00010000:00010000:1.0:1789760682.405859:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c8e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760682.405960:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515548928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47347us (47655us total) trans 4294971650 rc 0/0 00000100:00100000:1.0:1789760682.405970:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4362 00000100:00100000:1.0:1789760682.405979:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 660758947716 time 660823887409 state 0x2 ref 0 00000100:00100000:1.0:1789760682.409445:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760682.485790:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760682.618970:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971653 is committed 00000100:00100000:1.0:1789760682.626628:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760682.709810:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971654 is committed 00002000:00010000:1.0:1789760682.759917:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17768448 -> 17772544 00010000:00010000:1.0:1789760682.759936:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.759949:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.759958:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.759965:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760682.759969:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.759980:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.759999:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.760007:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1c908 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760682.760099:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515549568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36008us (38039us total) trans 4294971655 rc 0/0 00000100:00100000:1.0:1789760682.760110:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4367 00000100:00100000:1.0:1789760682.760118:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661092503455 time 661189363082 state 0x2 ref 0 00000100:00100000:1.0:1789760682.771042:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760682.809504:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971656 is committed 00000001:00080000:1.0:1789760682.883857:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971657 is committed 00000100:00100000:1.0:1789760682.952996:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760682.993570:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17784832 -> 17788928 00010000:00010000:1.0:1789760682.993585:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.993598:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.993608:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.993616:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760682.993621:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.993631:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.993648:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760682.993656:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1c924 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760682.993946:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515550080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40640us (40955us total) trans 4294971659 rc 0/0 00000100:00100000:1.0:1789760682.993957:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4371 00000100:00100000:1.0:1789760682.993968:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661357003497 time 661418576531 state 0x2 ref 0 00000001:00080000:1.0:1789760683.047347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971660 is committed 00000100:00100000:1.0:1789760683.057867:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760683.111673:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17793024 -> 17797120 00010000:00010000:1.0:1789760683.111688:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.111701:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.111736:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.111745:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760683.111750:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.111759:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.111789:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.111797:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1c932 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760683.111877:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515550336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53723us (54015us total) trans 4294971661 rc 0/0 00000100:00100000:1.0:1789760683.111886:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4373 00000100:00100000:1.0:1789760683.111895:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661465016345 time 661523427892 state 0x2 ref 0 00000100:00100000:1.0:1789760683.117033:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760683.201949:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971663 is committed 00000100:00100000:1.0:1789760683.207525:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760683.289127:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760683.407748:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971666 is committed 00000100:00100000:1.0:1789760683.412248:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515551104 00000100:00100000:1.0:1789760683.412258:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515551104 00000100:00100000:1.0:1789760683.412309:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661754861747 time 661801552514 state 0x0 ref 1 00000100:00100000:1.0:1789760683.412437:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661801812930 time 661801552514 state 0x1 ref 1 00000100:00100000:1.0:1789760683.412492:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661801812930 time 661877781031 state 0x0 ref 1 00000100:00100000:1.0:1789760683.412506:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515551104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760683.412553:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17817600, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x8807 00010000:00010000:1.0:1789760683.412561:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.412577:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=661s v=(0 0) 00010000:00010000:1.0:1789760683.412581:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760683.412588:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.412602:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760683.446931:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17817600 -> 17821696 00010000:00010000:1.0:1789760683.446946:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.446958:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.446968:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.446980:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760683.446985:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.446994:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.447013:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.447196:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c95c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760683.447346:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515551104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34831us (35316us total) trans 4294971667 rc 0/0 00000100:00100000:1.0:1789760683.447356:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4379 00000100:00100000:1.0:1789760683.447367:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661801812930 time 661877781031 state 0x2 ref 0 00000100:00100000:1.0:1789760683.455579:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760683.610240:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17829888 -> 17833984 00010000:00010000:1.0:1789760683.610256:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.610270:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.610284:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.610291:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760683.610296:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.610306:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.610338:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.610346:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1c971 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760683.610444:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515551488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52409us (52643us total) trans 4294971670 rc 0/0 00000100:00100000:1.0:1789760683.610454:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4382 00000100:00100000:1.0:1789760683.610463:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 661979625887 time 662023298881 state 0x2 ref 0 00000100:00100000:1.0:1789760683.619128:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760683.668616:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971671 is committed 00000001:00080000:1.0:1789760683.732289:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971672 is committed 00000100:00100000:1.0:1789760683.739596:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515551872 00000100:00100000:1.0:1789760683.739607:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515551872 00000100:00100000:1.0:1789760683.739660:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662085010075 time 662145070906 state 0x0 ref 1 00000100:00100000:1.0:1789760683.739675:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662145331322 time 662145070906 state 0x1 ref 1 00000100:00100000:1.0:1789760683.739744:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662145331322 time 662205032676 state 0x0 ref 1 00000100:00100000:1.0:1789760683.739758:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515551872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760683.739804:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17842176, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9007 00010000:00010000:1.0:1789760683.739813:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.739828:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=662s v=(0 0) 00010000:00010000:1.0:1789760683.739833:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760683.739840:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.739855:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760683.850800:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17842176 -> 17846272 00010000:00010000:1.0:1789760683.850821:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.850834:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.850847:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.850854:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760683.850858:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.850867:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.850887:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760683.850894:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1c986 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17842176->17846271] (req 17842176->17846271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760683.850983:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515551872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 111223us (111883us total) trans 4294971673 rc 0/0 00000100:00100000:1.0:1789760683.850992:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4385 00000100:00100000:1.0:1789760683.851002:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662145331322 time 662205032676 state 0x2 ref 0 00000001:00080000:1.0:1789760683.851042:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971673 is committed 00000100:00100000:1.0:1789760683.859690:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760683.966672:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971675 is committed 00000100:00100000:1.0:1789760684.017807:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760684.065399:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971677 is committed 00000100:00100000:1.0:1789760684.069011:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760684.125089:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17862656 -> 17866752 00010000:00010000:1.0:1789760684.125105:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.125119:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.125128:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.125135:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760684.125140:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.125150:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.125170:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.125176:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1c9a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760684.125264:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515552512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55949us (56259us total) trans 4294971678 rc 0/0 00000100:00100000:1.0:1789760684.125277:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4390 00000100:00100000:1.0:1789760684.125286:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662483587238 time 662534588189 state 0x2 ref 0 00000100:00100000:1.0:1789760684.131563:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760684.187453:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515552768 00000100:00100000:1.0:1789760684.187463:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515552768 00000100:00100000:1.0:1789760684.187519:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662534848605 time 662597083278 state 0x0 ref 1 00000100:00100000:1.0:1789760684.187536:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662597343694 time 662597083278 state 0x1 ref 1 00000100:00100000:1.0:1789760684.187563:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662597343694 time 662652852381 state 0x0 ref 1 00000100:00100000:1.0:1789760684.187575:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515552768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760684.187626:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17870848, mtime: 0x6aad94ac, atime: 0x0, ctime: 0x6aad94ac, blocks: 0x9007 00010000:00010000:1.0:1789760684.187636:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c9b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.187651:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=662s v=(0 0) 00010000:00010000:1.0:1789760684.187656:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760684.187662:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c9b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.187677:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1c9b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760684.494237:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17879040 -> 17883136 00010000:00010000:1.0:1789760684.494252:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.494264:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.494272:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.494279:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760684.494284:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.494293:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.494311:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.494318:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1c9c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760684.494567:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515553024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 128536us (128808us total) trans 4294971682 rc 0/0 00000100:00100000:1.0:1789760684.494578:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4394 00000100:00100000:1.0:1789760684.494588:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 662762841565 time 662831304345 state 0x2 ref 0 00000100:00100000:1.0:1789760684.501776:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760684.582955:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971683 is committed 00000100:00100000:1.0:1789760684.602031:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515553280 00000100:00100000:1.0:1789760684.602046:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515553280 00000001:00080000:1.0:1789760684.811060:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971685 is committed 00000100:00100000:1.0:1789760684.817046:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760684.995701:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515553792 00000100:00100000:1.0:1789760684.995867:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515553792 00000100:00100000:1.0:1789760684.995933:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663282930541 time 663352378474 state 0x0 ref 1 00000100:00100000:1.0:1789760684.995949:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663352638890 time 663352378474 state 0x1 ref 1 00000100:00100000:1.0:1789760684.995982:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663352638890 time 663461271115 state 0x0 ref 1 00000100:00100000:1.0:1789760684.995995:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515553792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760684.996047:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17903616, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0x9007 00010000:00010000:1.0:1789760684.996057:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1c9ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.996073:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=663s v=(0 0) 00010000:00010000:1.0:1789760684.996077:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760684.996084:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1c9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760684.996097:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1c9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17903616->17907711] (req 17903616->17907711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760685.058475:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760685.194871:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971690 is committed 00000001:00080000:1.0:1789760685.256703:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971691 is committed 00000100:00100000:1.0:1789760685.260639:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515554304 00000100:00100000:1.0:1789760685.260648:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515554304 00000100:00100000:1.0:1789760685.260698:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663615588518 time 663668714803 state 0x0 ref 1 00000100:00100000:1.0:1789760685.260747:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663668975219 time 663668714803 state 0x1 ref 1 00000100:00100000:1.0:1789760685.260769:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663668975219 time 663726058577 state 0x0 ref 1 00000100:00100000:1.0:1789760685.260782:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515554304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760685.260830:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17920000, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0x9007 00010000:00010000:1.0:1789760685.260838:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ca0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.260853:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=663s v=(0 0) 00010000:00010000:1.0:1789760685.260857:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760685.260863:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ca0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.260875:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ca0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760685.566579:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c25b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544400768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.566661:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760685.566736:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c259f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544401280:0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.566763:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760685.566794:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760685.566801:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c259880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544400896:0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.566822:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760685.566862:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544400768 00000100:00100000:1.0:1789760685.566870:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544400768 00000100:00100000:1.0:1789760685.566915:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544400896 00000100:00100000:1.0:1789760685.566918:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544400896 00000100:00100000:1.0:1789760685.566948:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789760685.566957:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544400768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.567020:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544400768:12345-0@lo:400:kworker.0 Request processed in 63us (358us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760685.567028:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789760685.567045:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1789760685.567051:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544400896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.567080:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544400896:12345-0@lo:400:kworker.0 Request processed in 29us (257us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760685.567086:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1789760685.567104:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544401280 00000100:00100000:1.0:1789760685.567107:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544401280 00000100:00100000:1.0:1789760685.567133:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:1.0:1789760685.567139:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544401280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.567169:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544401280:12345-0@lo:400:kworker.0 Request processed in 29us (406us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760685.567175:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:1.0:1789760685.567218:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c259880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544400896:0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.567229:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c25b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544400768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.567238:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c259f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544401280:0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.568983:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544400640 00000100:00100000:1.0:1789760685.568989:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544400640 00000100:00100000:1.0:1789760685.569117:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789760685.569126:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9092de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544400640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760685.569173:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9092de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544400640:12345-0@lo:400:kworker.0 Request processed in 49us (1138us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760685.569180:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789760685.600971:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515555072 00000100:00100000:1.0:1789760685.600980:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515555072 00000100:00100000:1.0:1789760685.601031:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 663963093852 time 664015274101 state 0x0 ref 1 00000100:00100000:1.0:1789760685.601046:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664015534517 time 664015274101 state 0x1 ref 1 00000100:00100000:1.0:1789760685.601075:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664015534517 time 664066364189 state 0x0 ref 1 00000100:00100000:1.0:1789760685.601087:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515555072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760685.601133:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17944576, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0x9007 00010000:00010000:1.0:1789760685.601141:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ca35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.601155:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=664s v=(0 0) 00010000:00010000:1.0:1789760685.601159:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760685.601167:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ca35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.601180:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ca35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17944576->17948671] (req 17944576->17948671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760685.717684:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515555328 00000100:00100000:1.0:1789760685.717693:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515555328 00000100:00100000:1.0:1789760685.717866:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664066624605 time 664140336632 state 0x0 ref 1 00000100:00100000:1.0:1789760685.717881:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664140597048 time 664140336632 state 0x1 ref 1 00000100:00100000:1.0:1789760685.717917:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664140597048 time 664183198908 state 0x0 ref 1 00000100:00100000:1.0:1789760685.717930:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515555328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760685.717976:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17952768, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0x9007 00010000:00010000:1.0:1789760685.717985:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ca43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.718001:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=664s v=(0 0) 00010000:00010000:1.0:1789760685.718005:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760685.718012:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ca43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.718026:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ca43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760685.976360:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 17965056 -> 17969152 00010000:00010000:1.0:1789760685.976376:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.976388:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.976407:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.976415:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760685.976419:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.976428:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.976445:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760685.976452:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ca58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760685.976546:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515555712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22967us (23191us total) trans 4294971703 rc 0/0 00000100:00100000:1.0:1789760685.976559:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4415 00000100:00100000:1.0:1789760685.976568:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664346544720 time 664418851676 state 0x2 ref 0 00000100:00100000:1.0:1789760686.034624:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515556352 00000100:00100000:1.0:1789760686.034637:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515556352 00000100:00100000:1.0:1789760686.034769:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664419112092 time 664446571036 state 0x0 ref 1 00000100:00100000:1.0:1789760686.034786:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664446831452 time 664446571036 state 0x1 ref 1 00000100:00100000:1.0:1789760686.034821:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 664446831452 time 664500109721 state 0x0 ref 1 00000100:00100000:1.0:1789760686.034835:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515556352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760686.034883:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 17973248, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0x9007 00010000:00010000:1.0:1789760686.034893:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ca66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.034909:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=664s v=(0 0) 00010000:00010000:1.0:1789760686.034913:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760686.034920:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ca66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760686.034934:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ca66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760686.527699:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760686.589250:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760686.846365:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760686.944564:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760687.143967:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760687.289806:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760687.337670:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760687.379783:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760687.421954:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760687.486201:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971722 is committed 00000100:00100000:1.0:1789760687.490066:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760687.518498:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971723 is committed 00000100:00100000:1.0:1789760687.524534:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760687.551826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971724 is committed 00000100:00100000:1.0:1789760687.555442:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760687.605106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971725 is committed 00000100:00100000:1.0:1789760687.616175:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544401408:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760687.616204:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1789760687.616910:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760687.616914:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760687.616936:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544401408:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760687.666416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971726 is committed 00000001:00080000:1.0:1789760687.706868:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971727 is committed 00002000:00010000:1.0:1789760687.706932:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18063360 -> 18067456 00010000:00010000:1.0:1789760687.707126:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.707141:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.707155:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.707162:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760687.707166:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.707175:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.707195:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.707203:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18063360->18067455] (req 18063360->18067455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760687.707294:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515559168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35909us (36405us total) trans 4294971727 rc 0/0 00000100:00100000:1.0:1789760687.707842:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4439 00000100:00100000:1.0:1789760687.707853:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666068033552 time 666136533552 state 0x2 ref 0 00000100:00100000:1.0:1789760687.711612:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760687.741274:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971728 is committed 00000100:00100000:1.0:1789760687.805603:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760687.898590:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515559936 00000100:00100000:1.0:1789760687.898600:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515559936 00000100:00100000:1.0:1789760687.898652:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666271360247 time 666319476022 state 0x0 ref 1 00000100:00100000:1.0:1789760687.898668:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666319736438 time 666319476022 state 0x1 ref 1 00000100:00100000:1.0:1789760687.898703:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666319736438 time 666363992637 state 0x0 ref 1 00000100:00100000:1.0:1789760687.898735:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515559936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760687.898781:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18087936, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:1.0:1789760687.898802:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cb2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.898819:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=666s v=(0 0) 00010000:00010000:1.0:1789760687.898823:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760687.898830:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cb2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760687.898844:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cb2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760687.930678:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971733 is committed 00000100:00100000:1.0:1789760688.002840:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00080000:1.0:1789760688.003110:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760688.035412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971736 is committed 00000100:00100000:1.0:1789760688.040409:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760688.040765:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=666s v=(0 0) 00010000:00010000:0.0:1789760688.040771:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.040778:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.040794:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760688.068613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971737 is committed 00002000:00010000:0.0:1789760688.068827:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18104320 -> 18108416 00010000:00010000:0.0:1789760688.068842:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.068853:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.068861:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.068868:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.068872:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.068881:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.068899:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.068907:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.069127:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515560448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28464us (28721us total) trans 4294971737 rc 0/0 00000100:00100000:0.0:1789760688.069137:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4449 00000100:00100000:0.0:1789760688.069147:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666468780944 time 666505935426 state 0x2 ref 0 00000100:00100000:0.0:1789760688.072760:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760688.103777:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18108416 -> 18112512 00010000:00010000:0.0:1789760688.103793:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.103807:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.103816:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.103823:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.103828:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.103837:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.103857:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.103869:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cb4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.103962:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515560576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30727us (31205us total) trans 4294971738 rc 0/0 00000100:00100000:0.0:1789760688.103971:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4450 00000100:00100000:0.0:1789760688.103990:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666506195842 time 666538507946 state 0x2 ref 0 00000100:00100000:1.0:1789760688.107449:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760688.107565:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515560704 00000100:00100000:0.0:1789760688.107574:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515560704 00000100:00100000:0.0:1789760688.107628:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666506195842 time 666538507946 state 0x0 ref 1 00000100:00100000:0.0:1789760688.107644:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666538768362 time 666538507946 state 0x1 ref 1 00000100:00100000:0.0:1789760688.107674:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666538768362 time 666572963164 state 0x0 ref 1 00000100:00100000:0.0:1789760688.107685:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515560704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.107910:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18112512, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:0.0:1789760688.107919:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.107946:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=666s v=(0 0) 00010000:00010000:0.0:1789760688.107950:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.107956:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.107974:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cb54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760688.169744:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760688.196769:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971740 is committed 00000100:00100000:0.0:1789760688.205567:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760688.253563:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18120704 -> 18124800 00010000:00010000:0.0:1789760688.253579:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.253593:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.253603:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.253610:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.253615:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.253625:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.253645:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.253654:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cb62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18120704->18124799] (req 18120704->18124799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.254571:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515560960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48590us (49006us total) trans 4294971741 rc 0/0 00000100:00100000:0.0:1789760688.254583:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4453 00000100:00100000:0.0:1789760688.254594:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666635655610 time 666671251955 state 0x2 ref 0 00002000:00010000:0.0:1789760688.302766:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18124800 -> 18128896 00010000:00010000:0.0:1789760688.302781:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.302795:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.302805:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.302812:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.302816:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.302825:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.302845:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.302857:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cb69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18124800->18128895] (req 18124800->18128895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.302947:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515561088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38827us (42414us total) trans 4294971742 rc 0/0 00000100:00100000:0.0:1789760688.302968:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4454 00000100:00100000:0.0:1789760688.302978:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666671512371 time 666729393103 state 0x2 ref 0 00000100:00100000:1.0:1789760688.309673:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760688.310085:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515561216 00000100:00100000:0.0:1789760688.310095:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515561216 00000100:00100000:0.0:1789760688.310151:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666671512371 time 666729393103 state 0x0 ref 1 00000100:00100000:0.0:1789760688.310166:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666729653519 time 666729393103 state 0x1 ref 1 00000100:00100000:0.0:1789760688.310238:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666729653519 time 666775526161 state 0x0 ref 1 00000100:00100000:0.0:1789760688.310253:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515561216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.310375:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18128896, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:0.0:1789760688.310400:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.310418:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=666s v=(0 0) 00010000:00010000:0.0:1789760688.310422:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.310428:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.310447:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760688.355137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971743 is committed 00002000:00010000:0.0:1789760688.356598:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18128896 -> 18132992 00010000:00010000:0.0:1789760688.356791:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.356805:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.356814:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.356820:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.356825:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.356833:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.356852:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.356860:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.357293:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515561216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47036us (47623us total) trans 4294971743 rc 0/0 00000100:00100000:0.0:1789760688.357314:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4455 00000100:00100000:0.0:1789760688.357325:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666729653519 time 666775526161 state 0x2 ref 0 00000100:00100000:0.0:1789760688.363285:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515561344 00000100:00100000:0.0:1789760688.363294:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515561344 00000100:00100000:0.0:1789760688.363343:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666729653519 time 666775526161 state 0x0 ref 1 00000100:00100000:0.0:1789760688.363359:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666775786577 time 666775526161 state 0x1 ref 1 00000100:00100000:0.0:1789760688.363400:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666775786577 time 666828688480 state 0x0 ref 1 00000100:00100000:0.0:1789760688.363411:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515561344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.363457:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18132992, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:0.0:1789760688.363465:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.363481:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=666s v=(0 0) 00010000:00010000:0.0:1789760688.363485:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.363491:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.363503:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cb77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18132992->18137087] (req 18132992->18137087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760688.407671:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971744 is committed 00000100:00100000:0.0:1789760688.416664:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515561472 00000100:00100000:0.0:1789760688.416676:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515561472 00000100:00100000:0.0:1789760688.416768:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666775786577 time 666828688480 state 0x0 ref 1 00000100:00100000:0.0:1789760688.416785:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666828948896 time 666828688480 state 0x1 ref 1 00000100:00100000:0.0:1789760688.416811:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666828948896 time 666882101125 state 0x0 ref 1 00000100:00100000:0.0:1789760688.416823:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515561472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.416871:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18137088, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:0.0:1789760688.416880:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.416902:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=666s v=(0 0) 00010000:00010000:0.0:1789760688.416907:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.416914:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.416928:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.447638:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760688.457510:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18137088 -> 18141184 00010000:00010000:0.0:1789760688.457532:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.457546:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.457555:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.457563:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.457569:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.457578:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.457600:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.457609:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cb7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18137088->18141183] (req 18137088->18141183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.457772:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515561472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40944us (41204us total) trans 4294971745 rc 0/0 00000100:00100000:0.0:1789760688.457783:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4457 00000100:00100000:0.0:1789760688.457794:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666828948896 time 666882101125 state 0x2 ref 0 00000001:00080000:0.0:1789760688.457838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971745 is committed 00000100:00100000:1.0:1789760688.468036:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760688.573767:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18141184 -> 18145280 00010000:00010000:0.0:1789760688.573788:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.573802:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.573811:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.573818:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.573823:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.573830:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.573852:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.573861:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cb85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18141184->18145279] (req 18141184->18145279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.573973:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515561600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 105704us (105942us total) trans 4294971746 rc 0/0 00000100:00100000:0.0:1789760688.573985:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4458 00000100:00100000:0.0:1789760688.573999:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 666882361541 time 666933540341 state 0x2 ref 0 00000001:00080000:1.0:1789760688.574568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971746 is committed 00000001:00080000:1.0:1789760688.632997:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971747 is committed 00000001:00080000:0.0:1789760688.680410:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971748 is committed 00000100:00100000:0.0:1789760688.690203:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760688.722179:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971749 is committed 00002000:00010000:0.0:1789760688.722291:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18153472 -> 18157568 00010000:00010000:0.0:1789760688.722308:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.722321:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.722331:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.722341:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.722346:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.722356:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.722388:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.722397:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cb9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18153472->18157567] (req 18153472->18157567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.722490:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515561984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31970us (32292us total) trans 4294971749 rc 0/0 00000100:00100000:0.0:1789760688.722501:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4461 00000100:00100000:0.0:1789760688.722511:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667110582223 time 667155790038 state 0x2 ref 0 00000100:00100000:0.0:1789760688.728962:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760688.729068:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515562112 00000100:00100000:1.0:1789760688.729078:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515562112 00000100:00100000:1.0:1789760688.729130:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667110582223 time 667155790038 state 0x0 ref 1 00000100:00100000:1.0:1789760688.729146:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667156050454 time 667155790038 state 0x1 ref 1 00000100:00100000:1.0:1789760688.729214:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667156050454 time 667194502573 state 0x0 ref 1 00000100:00100000:1.0:1789760688.729227:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515562112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760688.729273:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18157568, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:1.0:1789760688.729282:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.729310:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:1.0:1789760688.729315:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760688.729322:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.729337:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760688.783209:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971750 is committed 00002000:00010000:1.0:1789760688.783284:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18157568 -> 18161664 00010000:00010000:1.0:1789760688.783307:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.783321:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.783330:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.783337:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760688.783343:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.783353:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.783374:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760688.783394:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cba1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18157568->18161663] (req 18157568->18161663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760688.783492:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515562112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54262us (54534us total) trans 4294971750 rc 0/0 00000100:00100000:1.0:1789760688.783502:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4462 00000100:00100000:1.0:1789760688.783511:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667156050454 time 667194502573 state 0x2 ref 0 00000100:00100000:0.0:1789760688.799949:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515562240 00000100:00100000:0.0:1789760688.799960:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515562240 00000100:00100000:0.0:1789760688.800026:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667156050454 time 667194502573 state 0x0 ref 1 00000100:00100000:0.0:1789760688.800042:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667194762989 time 667194502573 state 0x1 ref 1 00000100:00100000:0.0:1789760688.800074:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667194762989 time 667265362680 state 0x0 ref 1 00000100:00100000:0.0:1789760688.800087:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515562240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.800135:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18161664, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:0.0:1789760688.800145:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.800159:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:0.0:1789760688.800164:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.800171:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.800186:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760688.913972:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18161664 -> 18165760 00010000:00010000:0.0:1789760688.913994:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.914008:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.914019:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.914027:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760688.914032:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.914039:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.914058:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.914066:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cba8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18161664->18165759] (req 18161664->18165759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.914162:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515562240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 114071us (114320us total) trans 4294971751 rc 0/0 00000100:00100000:0.0:1789760688.914173:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4463 00000100:00100000:0.0:1789760688.914181:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667194762989 time 667265362680 state 0x2 ref 0 00000100:00100000:0.0:1789760688.933981:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515562368 00000100:00100000:0.0:1789760688.933997:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515562368 00000100:00100000:0.0:1789760688.934072:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667194762989 time 667265362680 state 0x0 ref 1 00000100:00100000:0.0:1789760688.934088:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667265623096 time 667265362680 state 0x1 ref 1 00000100:00100000:0.0:1789760688.934125:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667265623096 time 667399414230 state 0x0 ref 1 00000100:00100000:0.0:1789760688.934139:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515562368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.934191:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18165760, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0x9007 00010000:00010000:0.0:1789760688.934200:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cbaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.934217:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:0.0:1789760688.934222:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.934229:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cbaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.934243:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cbaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.989552:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515562496 00000100:00100000:0.0:1789760688.989561:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515562496 00000100:00100000:0.0:1789760688.989676:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667265623096 time 667399414230 state 0x0 ref 1 00000100:00100000:0.0:1789760688.989692:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667399674646 time 667399414230 state 0x1 ref 1 00000100:00100000:0.0:1789760688.989815:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667399674646 time 667455102434 state 0x0 ref 1 00000100:00100000:0.0:1789760688.989826:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515562496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760688.989872:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18169856, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:0.0:1789760688.989880:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.989907:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:0.0:1789760688.989911:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.989919:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.989934:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.043144:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18169856 -> 18173952 00010000:00010000:0.0:1789760689.043165:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.043179:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.043188:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.043196:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760689.043201:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.043210:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.043228:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.043236:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cbb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.043337:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515562496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53506us (54092us total) trans 4294971753 rc 0/0 00000100:00100000:0.0:1789760689.043347:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4465 00000100:00100000:0.0:1789760689.043358:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667399674646 time 667455102434 state 0x2 ref 0 00000100:00100000:0.0:1789760689.048108:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667455362850 time 667513394972 state 0x0 ref 1 00000100:00100000:0.0:1789760689.048124:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515562624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760689.048173:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18173952, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:0.0:1789760689.048182:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.048197:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:0.0:1789760689.048203:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.048210:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.048224:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.080350:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18173952 -> 18178048 00010000:00010000:0.0:1789760689.080896:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.080913:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.080921:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.080928:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760689.080932:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.080942:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.080961:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.080969:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.081300:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515562624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33167us (33587us total) trans 4294971754 rc 0/0 00000100:00100000:0.0:1789760689.081311:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4466 00000100:00100000:0.0:1789760689.081320:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667455362850 time 667513394972 state 0x2 ref 0 00000100:00100000:1.0:1789760689.086538:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760689.128733:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971755 is committed 00000100:00100000:1.0:1789760689.132102:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515562880 00000100:00100000:1.0:1789760689.132112:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515562880 00000100:00100000:1.0:1789760689.132188:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667513655388 time 667552134923 state 0x0 ref 1 00000100:00100000:1.0:1789760689.132205:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667552395339 time 667552134923 state 0x1 ref 1 00000100:00100000:1.0:1789760689.132233:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667552395339 time 667597522951 state 0x0 ref 1 00000100:00100000:1.0:1789760689.132245:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515562880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760689.132300:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18182144, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:1.0:1789760689.132309:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cbcb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.132324:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:1.0:1789760689.132329:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760689.132335:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cbcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.132349:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cbcb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760689.187208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971756 is committed 00000100:00100000:0.0:1789760689.247783:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760689.307807:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515563264 00000100:00100000:0.0:1789760689.307816:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515563264 00000100:00100000:0.0:1789760689.307868:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667668786573 time 667713267399 state 0x0 ref 1 00000100:00100000:0.0:1789760689.307885:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667713527815 time 667713267399 state 0x1 ref 1 00000100:00100000:0.0:1789760689.307913:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667713527815 time 667773202538 state 0x0 ref 1 00000100:00100000:0.0:1789760689.307926:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cac450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515563264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760689.307971:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18194432, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:0.0:1789760689.307980:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbe0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.307996:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:0.0:1789760689.308000:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.308007:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.308032:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1cbe0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760689.343749:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971759 is committed 00000100:00100000:0.0:1789760689.348308:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760689.348782:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515563392 00000100:00100000:1.0:1789760689.348791:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515563392 00000001:00080000:1.0:1789760689.383006:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971760 is committed 00000100:00100000:0.0:1789760689.396427:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760689.474995:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760689.511376:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18202624 -> 18206720 00010000:00010000:0.0:1789760689.511396:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.511409:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.511418:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.511426:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760689.511430:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.511439:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.511459:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.511467:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cbee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18202624->18206719] (req 18202624->18206719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.511560:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515563520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 114895us (115139us total) trans 4294971761 rc 0/0 00000100:00100000:0.0:1789760689.511571:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4473 00000100:00100000:0.0:1789760689.511580:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667814531134 time 667861938777 state 0x2 ref 0 00000001:00080000:1.0:1789760689.512053:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971761 is committed 00000100:00100000:1.0:1789760689.516745:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515563648 00000100:00100000:1.0:1789760689.516757:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515563648 00000100:00100000:1.0:1789760689.516881:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667814531134 time 667861938777 state 0x0 ref 1 00000100:00100000:1.0:1789760689.516900:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667862199193 time 667861938777 state 0x1 ref 1 00000100:00100000:1.0:1789760689.516937:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667862199193 time 667982225751 state 0x0 ref 1 00000100:00100000:1.0:1789760689.516950:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515563648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760689.516996:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18206720, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:1.0:1789760689.517005:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cbf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.517021:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=667s v=(0 0) 00010000:00010000:1.0:1789760689.517025:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760689.517031:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cbf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.517045:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cbf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18206720->18210815] (req 18206720->18210815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760689.578431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971762 is committed 00000001:00080000:1.0:1789760689.642872:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971763 is committed 00002000:00010000:0.0:1789760689.643046:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18210816 -> 18214912 00010000:00010000:0.0:1789760689.643065:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.643077:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.643086:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.643093:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760689.643098:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.643107:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.643126:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.643140:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1cbfc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.643447:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515563776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58245us (59960us total) trans 4294971763 rc 0/0 00000100:00100000:0.0:1789760689.643458:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4475 00000100:00100000:0.0:1789760689.643468:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 667982486167 time 668050474151 state 0x2 ref 0 00000100:00100000:0.0:1789760689.647466:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760689.707299:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18214912 -> 18219008 00010000:00010000:0.0:1789760689.707318:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.707330:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.707338:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.707344:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760689.707348:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.707375:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.707400:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.707408:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1cc03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18214912->18219007] (req 18214912->18219007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.707498:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515563904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59708us (60038us total) trans 4294971764 rc 0/0 00000100:00100000:0.0:1789760689.707508:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4476 00000100:00100000:0.0:1789760689.707517:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668050734567 time 668113063698 state 0x2 ref 0 00000001:00080000:1.0:1789760689.708140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971764 is committed 00000001:00080000:1.0:1789760689.795617:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971765 is committed 00002000:00010000:0.0:1789760689.795777:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18219008 -> 18223104 00010000:00010000:0.0:1789760689.795792:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.795804:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.795881:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.795890:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760689.795895:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.795909:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.795928:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.795935:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1cc0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18219008->18223103] (req 18219008->18223103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.796026:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515564032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 78746us (79193us total) trans 4294971765 rc 0/0 00000100:00100000:0.0:1789760689.796036:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4477 00000100:00100000:0.0:1789760689.796047:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668113324114 time 668182554243 state 0x2 ref 0 00000100:00100000:0.0:1789760689.806610:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515564160 00000100:00100000:0.0:1789760689.806627:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515564160 00000100:00100000:0.0:1789760689.806694:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668113324114 time 668182554243 state 0x0 ref 1 00000100:00100000:0.0:1789760689.806723:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668182814659 time 668182554243 state 0x1 ref 1 00000100:00100000:0.0:1789760689.807747:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668182814659 time 668272993316 state 0x0 ref 1 00000100:00100000:0.0:1789760689.807772:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515564160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760689.807946:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18223104, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:0.0:1789760689.807955:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cc11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.807972:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:0.0:1789760689.807977:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.807984:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cc11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.807999:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cc11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18223104->18227199] (req 18223104->18227199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760689.873763:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971766 is committed 00000100:00100000:1.0:1789760689.880257:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760689.880910:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515564288 00000100:00100000:0.0:1789760689.880921:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515564288 00000100:00100000:0.0:1789760689.880977:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668182814659 time 668272993316 state 0x0 ref 1 00000100:00100000:0.0:1789760689.880993:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668273253732 time 668272993316 state 0x1 ref 1 00000100:00100000:0.0:1789760689.881026:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668273253732 time 668346315302 state 0x0 ref 1 00000100:00100000:0.0:1789760689.881039:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515564288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760689.881085:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18227200, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0x9007 00010000:00010000:0.0:1789760689.881094:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.881109:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:0.0:1789760689.881114:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.881121:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.881141:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760689.963817:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18227200 -> 18231296 00002000:00010000:1.0:1789760689.963824:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760689 -> 1789760690 00002000:00010000:1.0:1789760689.963827:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760689 -> 1789760690 00010000:00010000:1.0:1789760689.963840:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.963854:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.963863:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.963871:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760689.963875:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.963884:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.963903:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.963910:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cc18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18227200->18231295] (req 18227200->18231295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760689.964012:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515564288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 82972us (83760us total) trans 4294971767 rc 0/0 00000100:00100000:1.0:1789760689.964022:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4479 00000100:00100000:1.0:1789760689.964033:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668273253732 time 668346315302 state 0x2 ref 0 00000100:00100000:1.0:1789760689.970910:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515564416 00000100:00100000:1.0:1789760689.970922:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515564416 00000100:00100000:1.0:1789760689.970973:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668273253732 time 668346315302 state 0x0 ref 1 00000100:00100000:1.0:1789760689.970988:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668346575718 time 668346315302 state 0x1 ref 1 00000100:00100000:1.0:1789760689.971010:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668346575718 time 668436299302 state 0x0 ref 1 00000100:00100000:1.0:1789760689.971022:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515564416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760689.971070:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18231296, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:1.0:1789760689.971079:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cc1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.971095:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1789760689.971099:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760689.971106:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cc1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760689.971126:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cc1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760690.048138:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515564544 00000100:00100000:1.0:1789760690.048157:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515564544 00000100:00100000:0.0:1789760690.048338:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668346575718 time 668436299302 state 0x0 ref 1 00000100:00100000:0.0:1789760690.048355:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668436559718 time 668436299302 state 0x1 ref 1 00000100:00100000:0.0:1789760690.048391:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668436559718 time 668513679771 state 0x0 ref 1 00000100:00100000:0.0:1789760690.048405:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515564544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760690.048569:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18235392, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:0.0:1789760690.048578:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.048597:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:0.0:1789760690.048602:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.048609:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.048624:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760690.050546:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760690.131115:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18235392 -> 18239488 00010000:00010000:0.0:1789760690.131133:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.131148:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.131157:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.131165:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760690.131170:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.131179:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.131200:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.131207:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.131388:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515564544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 82981us (86989us total) trans 4294971769 rc 0/0 00000100:00100000:0.0:1789760690.131399:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4481 00000100:00100000:0.0:1789760690.131409:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668436559718 time 668513679771 state 0x2 ref 0 00000100:00100000:1.0:1789760690.154821:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760690.238546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971770 is committed 00000100:00100000:0.0:1789760690.242930:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515564800 00000100:00100000:0.0:1789760690.242940:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515564800 00000100:00100000:0.0:1789760690.242993:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668513940187 time 668622535807 state 0x0 ref 1 00000100:00100000:0.0:1789760690.243008:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668622796223 time 668622535807 state 0x1 ref 1 00000100:00100000:0.0:1789760690.243401:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668622796223 time 668708688046 state 0x0 ref 1 00000100:00100000:0.0:1789760690.243416:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515564800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760690.243476:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18243584, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:0.0:1789760690.243485:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.243502:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:0.0:1789760690.243506:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.243513:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.243532:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760690.281163:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971771 is committed 00000100:00100000:1.0:1789760690.332523:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760690.332626:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515565056 00000100:00100000:0.0:1789760690.332635:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515565056 00000100:00100000:0.0:1789760690.332698:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668708948462 time 668752689002 state 0x0 ref 1 00000100:00100000:0.0:1789760690.332922:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668752949418 time 668752689002 state 0x1 ref 1 00000100:00100000:0.0:1789760690.332958:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668752949418 time 668798247641 state 0x0 ref 1 00000100:00100000:0.0:1789760690.332971:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515565056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760690.333017:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18251776, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:0.0:1789760690.333026:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.333041:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:0.0:1789760690.333046:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.333052:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.333066:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cc42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18251776->18255871] (req 18251776->18255871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760690.377741:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971773 is committed 00000100:00100000:1.0:1789760690.385143:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515565184 00000100:00100000:1.0:1789760690.385153:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515565184 00000100:00100000:1.0:1789760690.385208:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668752949418 time 668798247641 state 0x0 ref 1 00000100:00100000:1.0:1789760690.385225:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668798508057 time 668798247641 state 0x1 ref 1 00000100:00100000:1.0:1789760690.385260:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668798508057 time 668850549215 state 0x0 ref 1 00000100:00100000:1.0:1789760690.385273:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515565184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760690.385330:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18255872, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:1.0:1789760690.385339:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cc49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760690.385356:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1789760690.385361:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760690.385376:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cc49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760690.385390:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cc49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760690.427987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971774 is committed 00000100:00100000:0.0:1789760690.432409:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515565312 00000100:00100000:0.0:1789760690.432420:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515565312 00000100:00100000:0.0:1789760690.432472:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668798508057 time 668850549215 state 0x0 ref 1 00000100:00100000:0.0:1789760690.432486:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668850809631 time 668850549215 state 0x1 ref 1 00000100:00100000:0.0:1789760690.432517:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668850809631 time 668897805794 state 0x0 ref 1 00000100:00100000:0.0:1789760690.432530:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515565312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760690.432580:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18259968, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:0.0:1789760690.432588:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.432608:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=668s v=(0 0) 00010000:00010000:0.0:1789760690.432612:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.432618:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.432631:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.493842:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760690.509501:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18259968 -> 18264064 00010000:00010000:0.0:1789760690.509517:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.509530:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.509540:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.509549:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760690.509554:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.509564:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.509589:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.509596:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cc50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.509686:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515565312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 77154us (77393us total) trans 4294971775 rc 0/0 00000100:00100000:0.0:1789760690.509697:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4487 00000100:00100000:0.0:1789760690.509749:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 668850809631 time 668897805794 state 0x2 ref 0 00000001:00080000:1.0:1789760690.642073:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971777 is committed 00000100:00100000:0.0:1789760690.686505:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544402048:0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.686541:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760690.686608:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544402048 00000100:00100000:0.0:1789760690.686613:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544402048 00000100:00100000:0.0:1789760690.686649:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:0.0:1789760690.686656:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544402048:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.686696:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544402048:12345-0@lo:400:kworker.0 Request processed in 40us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760690.686703:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00080000:0.0:1789760690.686917:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760690.686950:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544402048:0@lo:400:kworker.0 00000001:00080000:0.0:1789760690.689208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971778 is committed 00000100:00100000:0.0:1789760690.689809:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be32a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544401792:0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.689880:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760690.689918:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760690.689926:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544401920:0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.689949:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760690.689976:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544401792 00000100:00100000:0.0:1789760690.689983:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544401792 00000100:00100000:0.0:1789760690.690032:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544401920 00000100:00100000:0.0:1789760690.690036:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544401920 00000100:00100000:0.0:1789760690.690072:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789760690.690080:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544401792:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.690131:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544401792:12345-0@lo:400:kworker.0 Request processed in 51us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760690.690138:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1789760690.690153:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:0.0:1789760690.690159:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544401920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.690186:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544401920:12345-0@lo:400:kworker.0 Request processed in 28us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760690.690192:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00080000:0.0:1789760690.690212:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760690.690244:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544401920:0@lo:400:kworker.0 00000100:00100000:0.0:1789760690.690254:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be32a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544401792:0@lo:400:kworker.0 00000100:00100000:1.0:1789760690.692469:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760690.692486:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544402304:0@lo:400:kworker.0 00000100:00100000:1.0:1789760690.692536:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760690.692991:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544402176 00000100:00100000:1.0:1789760690.692999:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544402176 00000100:00100000:1.0:1789760690.693184:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760690.693193:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544402176:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760690.693242:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544402176:12345-0@lo:400:kworker.0 Request processed in 51us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760690.693249:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760690.698427:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515565824 00000100:00100000:1.0:1789760690.698439:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515565824 00000100:00100000:1.0:1789760690.698495:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669069951991 time 669113959105 state 0x0 ref 1 00000100:00100000:1.0:1789760690.698510:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669114219521 time 669113959105 state 0x1 ref 1 00000100:00100000:1.0:1789760690.698582:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669114219521 time 669163871080 state 0x0 ref 1 00000100:00100000:1.0:1789760690.698599:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515565824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760690.698652:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18276352, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:1.0:1789760690.698661:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cc6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760690.698677:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=669s v=(0 0) 00010000:00010000:1.0:1789760690.698681:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760690.698688:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cc6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760690.698703:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cc6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760690.749630:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760690.826310:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515566080 00000100:00100000:0.0:1789760690.826319:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515566080 00000100:00100000:0.0:1789760690.826379:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669164131496 time 669215693867 state 0x0 ref 1 00000100:00100000:0.0:1789760690.826394:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669215954283 time 669215693867 state 0x1 ref 1 00000100:00100000:0.0:1789760690.826422:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669215954283 time 669291710668 state 0x0 ref 1 00000100:00100000:0.0:1789760690.826434:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515566080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760690.826479:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18284544, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0x9007 00010000:00010000:0.0:1789760690.826488:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cc7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.826502:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760690.826507:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.826514:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cc7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.826529:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cc7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18284544->18288639] (req 18284544->18288639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760690.899506:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760690.966491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971782 is committed 00000001:00080000:0.0:1789760691.058542:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971783 is committed 00000100:00100000:0.0:1789760691.062434:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760691.092727:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971784 is committed 00000100:00100000:1.0:1789760691.097608:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760691.133951:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760691.133998:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760691.134022:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760691.134058:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515566720 00000100:00100000:0.0:1789760691.134067:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515566720 00000100:00100000:0.0:1789760691.134124:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 372 00000100:00100000:0.0:1789760691.134133:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9092df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515566720:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.134205:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9092df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515566720:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 73us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.134212:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 372 00000100:00100000:0.0:1789760691.134244:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515566848 00000100:00100000:0.0:1789760691.134248:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515566848 00000100:00100000:0.0:1789760691.134281:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515566976 00000100:00100000:0.0:1789760691.134284:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515566976 00000100:00100000:0.0:1789760691.134307:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 406 00000100:00100000:0.0:1789760691.134313:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515566848:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.134345:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515566848:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 32us (405us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.134350:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 406 00000100:00100000:0.0:1789760691.134374:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 407 00000100:00100000:0.0:1789760691.134381:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515566976:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760691.134405:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515566976:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 26us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760691.134411:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 407 00000100:00080000:0.0:1789760691.134429:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:1.0:1789760691.150501:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18300928 -> 18305024 00010000:00010000:1.0:1789760691.150520:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.150534:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.150544:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.150551:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760691.150556:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.150572:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.150590:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.150597:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1cc96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.150688:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515566592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51877us (53085us total) trans 4294971785 rc 0/0 00000100:00100000:1.0:1789760691.150699:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4497 00000100:00100000:1.0:1789760691.150741:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669528312678 time 669564083161 state 0x2 ref 0 00000100:00100000:0.0:1789760691.157589:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760691.203932:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971786 is committed 00000100:00100000:0.0:1789760691.212565:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515567232 00000100:00100000:0.0:1789760691.212575:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515567232 00000100:00100000:0.0:1789760691.212628:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669564343577 time 669623121538 state 0x0 ref 1 00000100:00100000:0.0:1789760691.212659:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669623381954 time 669623121538 state 0x1 ref 1 00000100:00100000:0.0:1789760691.212692:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669623381954 time 669677981204 state 0x0 ref 1 00000100:00100000:0.0:1789760691.212739:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806191c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515567232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760691.212788:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18309120, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9007 00010000:00010000:0.0:1789760691.212797:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cca4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.212813:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760691.212817:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.212824:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.212837:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18309120->18313215] (req 18309120->18313215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760691.311906:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18313216 -> 18317312 00010000:00010000:1.0:1789760691.311922:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.311936:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.311945:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.311958:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760691.311963:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.311973:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.311992:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.312013:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ccab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.312453:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515567360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53565us (53787us total) trans 4294971788 rc 0/0 00000100:00100000:1.0:1789760691.312465:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4500 00000100:00100000:1.0:1789760691.312474:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669678241620 time 669724161359 state 0x2 ref 0 00000100:00100000:0.0:1789760691.330469:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760691.330892:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669724421775 time 669796177698 state 0x0 ref 1 00000100:00100000:0.0:1789760691.330905:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515567488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760691.330955:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18317312, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9007 00010000:00010000:0.0:1789760691.330964:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.330979:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1789760691.330983:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.330988:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.331133:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760691.402072:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18317312 -> 18321408 00010000:00010000:0.0:1789760691.402094:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.402108:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.402117:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.402125:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760691.402130:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.402139:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.402157:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.402165:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ccb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.402271:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515567488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71358us (71802us total) trans 4294971789 rc 0/0 00000100:00100000:0.0:1789760691.402455:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4501 00000100:00100000:0.0:1789760691.402465:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 669724421775 time 669796177698 state 0x2 ref 0 00000001:00080000:0.0:1789760691.480320:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971790 is committed 00000100:00100000:1.0:1789760691.498937:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760691.518761:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760691.568987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971791 is committed 00000100:00100000:1.0:1789760691.580235:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760691.623788:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971792 is committed 00000100:00100000:0.0:1789760691.627857:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760691.660343:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18333696 -> 18337792 00010000:00010000:1.0:1789760691.660373:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.660392:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.660401:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.660408:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760691.660412:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.660421:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.660440:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.660448:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ccce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.660530:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515568000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32437us (32676us total) trans 4294971793 rc 0/0 00000100:00100000:1.0:1789760691.660541:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4505 00000100:00100000:1.0:1789760691.660550:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670046200200 time 670093361644 state 0x2 ref 0 00000100:00100000:1.0:1789760691.664278:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760691.696927:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971794 is committed 00000100:00100000:1.0:1789760691.701246:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760691.701393:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515568256 00000100:00100000:0.0:1789760691.701403:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515568256 00000100:00100000:0.0:1789760691.701456:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670093622060 time 670130401220 state 0x0 ref 1 00000100:00100000:0.0:1789760691.701470:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670130661636 time 670130401220 state 0x1 ref 1 00000100:00100000:0.0:1789760691.701499:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670130661636 time 670166787985 state 0x0 ref 1 00000100:00100000:0.0:1789760691.701511:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515568256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760691.701560:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18341888, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9007 00010000:00010000:0.0:1789760691.701570:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.701585:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760691.701589:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.701594:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.701608:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760691.744179:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18341888 -> 18345984 00010000:00010000:1.0:1789760691.744194:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.744208:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.744218:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.744226:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760691.744230:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.744238:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.744256:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.744272:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ccdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.744491:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515568256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42976us (43247us total) trans 4294971795 rc 0/0 00000100:00100000:1.0:1789760691.744501:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4507 00000100:00100000:1.0:1789760691.744511:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670130661636 time 670166787985 state 0x2 ref 0 00000100:00100000:1.0:1789760691.891227:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760691.900906:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18354176, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0x9007 00010000:00010000:0.0:1789760691.900923:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ccf1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.900941:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=670s v=(0 0) 00010000:00010000:0.0:1789760691.900946:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.900953:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ccf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.900968:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ccf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.990571:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515568768 00000100:00100000:1.0:1789760691.990581:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515568768 00000100:00100000:1.0:1789760691.990635:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670263866586 time 670357953830 state 0x0 ref 1 00000100:00100000:1.0:1789760691.990651:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670358214246 time 670357953830 state 0x1 ref 1 00000100:00100000:1.0:1789760691.990683:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670358214246 time 670455972438 state 0x0 ref 1 00000100:00100000:1.0:1789760691.990695:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515568768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760691.990788:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18358272, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:1.0:1789760691.990797:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ccf8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.990812:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=670s v=(0 0) 00010000:00010000:1.0:1789760691.990817:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760691.990823:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ccf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.990836:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ccf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.055807:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515568896 00000100:00100000:1.0:1789760692.055817:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515568896 00000100:00100000:1.0:1789760692.055884:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670358214246 time 670455972438 state 0x0 ref 1 00000100:00100000:1.0:1789760692.055903:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670456232854 time 670455972438 state 0x1 ref 1 00000100:00100000:1.0:1789760692.055929:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670456232854 time 670521218162 state 0x0 ref 1 00000100:00100000:1.0:1789760692.055941:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515568896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760692.055988:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18362368, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:1.0:1789760692.056008:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.056024:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=670s v=(0 0) 00010000:00010000:1.0:1789760692.056029:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.056045:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.056059:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760692.056267:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760692.170845:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18362368 -> 18366464 00010000:00010000:1.0:1789760692.170862:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.170875:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.170884:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.170896:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760692.170900:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.170909:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.170928:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.170935:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ccff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18362368->18366463] (req 18362368->18366463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.171015:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515568896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 115072us (115336us total) trans 4294971800 rc 0/0 00000100:00100000:1.0:1789760692.171025:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4512 00000100:00100000:1.0:1789760692.171048:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670456232854 time 670521218162 state 0x2 ref 0 00000100:00100000:0.0:1789760692.177110:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760692.177198:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515569024 00000100:00100000:1.0:1789760692.177208:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515569024 00000100:00100000:1.0:1789760692.177283:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670456232854 time 670521218162 state 0x0 ref 1 00000100:00100000:1.0:1789760692.177299:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670521478578 time 670521218162 state 0x1 ref 1 00000100:00100000:1.0:1789760692.177327:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670521478578 time 670642615661 state 0x0 ref 1 00000100:00100000:1.0:1789760692.177343:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515569024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760692.177403:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18366464, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:1.0:1789760692.177412:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.177427:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=670s v=(0 0) 00010000:00010000:1.0:1789760692.177432:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.177438:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.177451:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760692.227248:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18366464 -> 18370560 00010000:00010000:1.0:1789760692.227472:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.227555:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.227565:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.227572:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760692.227577:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.227587:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.227609:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.227616:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.228154:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515569024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50365us (50683us total) trans 4294971801 rc 0/0 00000100:00100000:1.0:1789760692.228167:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4513 00000100:00100000:1.0:1789760692.228178:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670521478578 time 670642615661 state 0x2 ref 0 00000001:00080000:1.0:1789760692.446810:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971803 is committed 00002000:00010000:1.0:1789760692.446875:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18374656 -> 18378752 00010000:00010000:1.0:1789760692.446890:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.446903:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.446913:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.446925:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760692.446930:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.446939:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.446958:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.446966:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cd14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.447062:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515569280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 90484us (91220us total) trans 4294971803 rc 0/0 00000100:00100000:1.0:1789760692.447073:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4515 00000100:00100000:1.0:1789760692.447084:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670719183069 time 670821847833 state 0x2 ref 0 00000100:00100000:0.0:1789760692.453039:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760692.538687:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515569536 00000100:00100000:1.0:1789760692.538697:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515569536 00000100:00100000:1.0:1789760692.538764:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670822108249 time 670918578208 state 0x0 ref 1 00000100:00100000:1.0:1789760692.538779:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670918838624 time 670918578208 state 0x1 ref 1 00000100:00100000:1.0:1789760692.538811:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670918838624 time 671004099929 state 0x0 ref 1 00000100:00100000:1.0:1789760692.538825:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515569536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760692.538874:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18382848, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:1.0:1789760692.538883:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.538899:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760692.538904:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.538910:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.538928:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.542168:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760692.592280:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971805 is committed 00002000:00010000:1.0:1789760692.592679:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18382848 -> 18386944 00010000:00010000:1.0:1789760692.592695:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.593864:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.593876:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.593884:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760692.593890:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.593902:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.593924:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.593932:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.594090:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515569536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55260us (55688us total) trans 4294971805 rc 0/0 00000100:00100000:1.0:1789760692.594102:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4517 00000100:00100000:1.0:1789760692.594111:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670918838624 time 671004099929 state 0x2 ref 0 00000100:00100000:1.0:1789760692.600655:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515569664 00000100:00100000:1.0:1789760692.600666:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515569664 00000100:00100000:1.0:1789760692.600758:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 670918838624 time 671004099929 state 0x0 ref 1 00000100:00100000:1.0:1789760692.600773:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671004360345 time 671004099929 state 0x1 ref 1 00000100:00100000:1.0:1789760692.600804:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671004360345 time 671066093202 state 0x0 ref 1 00000100:00100000:1.0:1789760692.600817:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515569664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760692.600866:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18386944, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:1.0:1789760692.600875:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.600892:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760692.600897:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.600903:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.600917:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760692.675386:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18386944 -> 18391040 00010000:00010000:1.0:1789760692.675405:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.675419:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.675427:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.675434:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760692.675439:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.675448:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.675468:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.675476:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.675563:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515569664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74744us (75056us total) trans 4294971806 rc 0/0 00000100:00100000:1.0:1789760692.675574:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4518 00000100:00100000:1.0:1789760692.675586:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671004360345 time 671066093202 state 0x2 ref 0 00000001:00080000:0.0:1789760692.676502:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971806 is committed 00000100:00100000:0.0:1789760692.680984:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760692.682462:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515569792 00000100:00100000:1.0:1789760692.682474:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515569792 00000100:00100000:0.0:1789760692.682608:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671004360345 time 671066093202 state 0x0 ref 1 00000100:00100000:0.0:1789760692.682624:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671066353618 time 671066093202 state 0x1 ref 1 00000100:00100000:0.0:1789760692.682662:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671066353618 time 671147950978 state 0x0 ref 1 00000100:00100000:0.0:1789760692.682675:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515569792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760692.682743:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18391040, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:0.0:1789760692.682753:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cd30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.682769:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760692.682773:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.682780:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cd30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.682794:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1cd30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.738326:0:11703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544402560 00000100:00100000:0.0:1789760692.738334:0:11703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544402560 00000100:00100000:0.0:1789760692.738399:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:0.0:1789760692.738424:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544402560:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760692.738452:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760692.760467:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544402560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 22034us (22236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.760483:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000001:00080000:1.0:1789760692.790111:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971807 is committed 00000100:00100000:0.0:1789760692.791987:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515569920 00000100:00100000:0.0:1789760692.791998:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515569920 00000100:00100000:0.0:1789760692.792053:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671066353618 time 671147950978 state 0x0 ref 1 00000100:00100000:0.0:1789760692.792069:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671148211394 time 671147950978 state 0x1 ref 1 00000100:00100000:0.0:1789760692.792104:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671148211394 time 671257393455 state 0x0 ref 1 00000100:00100000:0.0:1789760692.792116:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515569920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760692.792164:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18395136, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0x9007 00010000:00010000:0.0:1789760692.792173:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.792189:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760692.792194:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.792201:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.792215:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760692.859076:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18395136 -> 18399232 00010000:00010000:1.0:1789760692.859095:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.859108:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.859116:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.859122:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760692.859127:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.859136:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.859158:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.859165:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cd37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.859259:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515569920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67140us (67557us total) trans 4294971808 rc 0/0 00000100:00100000:1.0:1789760692.859278:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4520 00000100:00100000:1.0:1789760692.859287:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671148211394 time 671257393455 state 0x2 ref 0 00000001:00080000:1.0:1789760692.859329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971808 is committed 00000100:00100000:0.0:1789760692.954851:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760693.024968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971810 is committed 00002000:00010000:1.0:1789760693.025136:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18403328 -> 18407424 00010000:00010000:1.0:1789760693.025152:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.025165:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.025174:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.025182:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760693.025187:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.025201:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.025220:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.025227:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cd45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.025334:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515570176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70260us (70487us total) trans 4294971810 rc 0/0 00000100:00100000:1.0:1789760693.025353:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4522 00000100:00100000:1.0:1789760693.025364:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671331673012 time 671420346584 state 0x2 ref 0 00000100:00100000:1.0:1789760693.038882:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515570304 00000100:00100000:1.0:1789760693.038894:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515570304 00000100:00100000:1.0:1789760693.038960:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671331673012 time 671420346584 state 0x0 ref 1 00000100:00100000:1.0:1789760693.038977:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671420607000 time 671420346584 state 0x1 ref 1 00000100:00100000:1.0:1789760693.039012:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671420607000 time 671504300912 state 0x0 ref 1 00000100:00100000:1.0:1789760693.039024:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515570304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760693.039075:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18407424, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:1.0:1789760693.039085:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.039104:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760693.039108:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.039114:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.039128:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1cd4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760693.105235:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971811 is committed 00000100:00100000:1.0:1789760693.115184:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515570432 00000100:00100000:1.0:1789760693.115196:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515570432 00000100:00100000:1.0:1789760693.115251:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671420607000 time 671504300912 state 0x0 ref 1 00000100:00100000:1.0:1789760693.115268:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671504561328 time 671504300912 state 0x1 ref 1 00000100:00100000:1.0:1789760693.115302:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671504561328 time 671580591206 state 0x0 ref 1 00000100:00100000:1.0:1789760693.115315:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515570432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760693.115382:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18411520, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:1.0:1789760693.115391:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.115407:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760693.115411:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.115417:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.115454:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.217300:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18411520 -> 18415616 00010000:00010000:0.0:1789760693.217316:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.217329:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.217521:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.217531:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.217537:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.217547:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.217567:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.217575:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1cd53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.222764:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515570432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 107439us (108553us total) trans 4294971812 rc 0/0 00000100:00100000:0.0:1789760693.222778:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4524 00000100:00100000:0.0:1789760693.222788:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671504561328 time 671580591206 state 0x2 ref 0 00000100:00100000:0.0:1789760693.228631:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760693.287243:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18415616 -> 18419712 00010000:00010000:0.0:1789760693.287257:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.287419:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.287429:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.287436:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.287441:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.287449:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.287468:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.287477:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cd5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.287599:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515570560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58303us (58973us total) trans 4294971813 rc 0/0 00000100:00100000:0.0:1789760693.287610:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4525 00000100:00100000:0.0:1789760693.287619:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671580851622 time 671694570679 state 0x2 ref 0 00000100:00100000:0.0:1789760693.296784:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515570688 00000100:00100000:0.0:1789760693.296793:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515570688 00000100:00100000:0.0:1789760693.296839:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671580851622 time 671694570679 state 0x0 ref 1 00000100:00100000:0.0:1789760693.296853:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671694831095 time 671694570679 state 0x1 ref 1 00000100:00100000:0.0:1789760693.296874:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671694831095 time 671762163574 state 0x0 ref 1 00000100:00100000:0.0:1789760693.296885:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515570688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760693.296926:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18419712, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:0.0:1789760693.296935:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.296949:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:0.0:1789760693.296954:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.296960:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.296974:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.345423:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18419712 -> 18423808 00010000:00010000:0.0:1789760693.345453:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.345467:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.345482:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.345490:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.345495:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.345503:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.345523:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.345531:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1cd61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.345632:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515570688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48742us (48959us total) trans 4294971814 rc 0/0 00000100:00100000:0.0:1789760693.345642:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4526 00000100:00100000:0.0:1789760693.345652:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671694831095 time 671762163574 state 0x2 ref 0 00000001:00080000:1.0:1789760693.397568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971815 is committed 00002000:00010000:0.0:1789760693.397751:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18423808 -> 18427904 00010000:00010000:0.0:1789760693.397770:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.397793:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.397805:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.397813:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.397817:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.397832:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.397855:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.397864:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cd68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.397994:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515570816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44718us (45202us total) trans 4294971815 rc 0/0 00000100:00100000:0.0:1789760693.398004:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4527 00000100:00100000:0.0:1789760693.398013:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671762423990 time 671818549146 state 0x2 ref 0 00000100:00100000:1.0:1789760693.407936:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515570944 00000100:00100000:1.0:1789760693.407952:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515570944 00000100:00100000:1.0:1789760693.408022:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671762423990 time 671818549146 state 0x0 ref 1 00000100:00100000:1.0:1789760693.408039:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671818809562 time 671818549146 state 0x1 ref 1 00000100:00100000:1.0:1789760693.408140:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671818809562 time 671873428241 state 0x0 ref 1 00000100:00100000:1.0:1789760693.408154:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cad450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515570944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760693.408202:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18427904, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:1.0:1789760693.408211:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.408228:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760693.408232:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.408238:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.408252:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.456741:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18427904 -> 18432000 00010000:00010000:0.0:1789760693.456757:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.456769:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.456778:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.456894:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.456900:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.456910:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.456931:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.456938:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cd6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.460954:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515571072 00000100:00100000:1.0:1789760693.460965:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515571072 00000100:00100000:1.0:1789760693.461019:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671818809562 time 671873428241 state 0x0 ref 1 00000100:00100000:1.0:1789760693.461036:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671873688657 time 671873428241 state 0x1 ref 1 00000100:00100000:1.0:1789760693.461065:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671873688657 time 671926353640 state 0x0 ref 1 00000100:00100000:1.0:1789760693.461078:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760693.461126:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18432000, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:1.0:1789760693.461135:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.461150:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760693.461154:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.461160:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.461172:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.505276:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18432000 -> 18436096 00010000:00010000:0.0:1789760693.505292:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.505305:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.505315:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.505322:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.505327:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.505335:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.505371:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.505386:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cd76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18432000->18436095] (req 18432000->18436095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.505478:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44397us (44812us total) trans 4294971817 rc 0/0 00000100:00100000:0.0:1789760693.505487:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4529 00000100:00100000:0.0:1789760693.505497:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671873688657 time 671926353640 state 0x2 ref 0 00000100:00100000:0.0:1789760693.517103:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760693.517217:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515571200 00000100:00100000:1.0:1789760693.517231:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515571200 00000100:00100000:1.0:1789760693.517288:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671873688657 time 671926353640 state 0x0 ref 1 00000100:00100000:1.0:1789760693.517303:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671926614056 time 671926353640 state 0x1 ref 1 00000100:00100000:1.0:1789760693.517335:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671926614056 time 671982623904 state 0x0 ref 1 00000100:00100000:1.0:1789760693.517363:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760693.517569:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18436096, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:1.0:1789760693.517579:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.517595:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=671s v=(0 0) 00010000:00010000:1.0:1789760693.517600:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.517613:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.517634:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.566353:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760693.571649:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18436096 -> 18440192 00010000:00010000:0.0:1789760693.571665:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.571679:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.571689:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.571697:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.571702:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.571749:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.571771:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.571780:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cd7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.573857:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56493us (56753us total) trans 4294971818 rc 0/0 00000100:00100000:1.0:1789760693.573872:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4530 00000100:00100000:1.0:1789760693.573882:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671926614056 time 671982623904 state 0x2 ref 0 00000100:00100000:0.0:1789760693.584500:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515571328 00000100:00100000:0.0:1789760693.584512:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515571328 00000100:00100000:0.0:1789760693.584566:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671926614056 time 671982623904 state 0x0 ref 1 00000100:00100000:0.0:1789760693.584583:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671982884320 time 671982623904 state 0x1 ref 1 00000100:00100000:0.0:1789760693.584616:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671982884320 time 672049904667 state 0x0 ref 1 00000100:00100000:0.0:1789760693.584629:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515571328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760693.584679:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18440192, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:0.0:1789760693.584687:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cd84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.584703:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.584738:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.584746:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cd84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.584760:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cd84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.696577:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515571456 00000100:00100000:0.0:1789760693.696587:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515571456 00000100:00100000:0.0:1789760693.696640:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 671982884320 time 672049904667 state 0x0 ref 1 00000100:00100000:0.0:1789760693.696656:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672050165083 time 672049904667 state 0x1 ref 1 00000100:00100000:0.0:1789760693.696693:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672050165083 time 672161981924 state 0x0 ref 1 00000100:00100000:0.0:1789760693.696735:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760693.696786:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18444288, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0x9007 00010000:00010000:0.0:1789760693.696795:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.696817:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.696822:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.696827:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.696840:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.787066:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18444288 -> 18448384 00010000:00010000:0.0:1789760693.787087:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.787099:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.787109:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.787117:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760693.787121:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.787130:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.787152:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.787161:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1cd8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18444288->18448383] (req 18444288->18448383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.787397:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 90688us (90904us total) trans 4294971820 rc 0/0 00000100:00100000:0.0:1789760693.787408:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4532 00000100:00100000:0.0:1789760693.787417:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672050165083 time 672161981924 state 0x2 ref 0 00000100:00100000:0.0:1789760693.801232:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760693.900174:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971821 is committed 00002000:00010000:1.0:1789760693.900400:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18448384 -> 18452480 00010000:00010000:1.0:1789760693.900416:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.900429:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.900439:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.900445:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760693.900451:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.900460:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.900480:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.900488:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cd92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.900579:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515571584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 99107us (99352us total) trans 4294971821 rc 0/0 00000100:00100000:1.0:1789760693.900595:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4533 00000100:00100000:1.0:1789760693.900605:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672162242340 time 672266747817 state 0x2 ref 0 00000100:00100000:0.0:1789760693.908648:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760694.033574:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18452480 -> 18456576 00002000:00010000:0.0:1789760694.033584:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760693 -> 1789760694 00002000:00010000:0.0:1789760694.033587:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760693 -> 1789760694 00010000:00010000:0.0:1789760694.033600:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.033613:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.033622:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.033630:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760694.033635:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.033644:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.033663:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.033671:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cd99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.033810:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515571712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 124839us (125167us total) trans 4294971822 rc 0/0 00000100:00100000:0.0:1789760694.033821:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4534 00000100:00100000:0.0:1789760694.033830:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672267008233 time 672374243660 state 0x2 ref 0 00000001:00080000:0.0:1789760694.033866:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971822 is committed 00000100:00100000:1.0:1789760694.049094:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515571840 00000100:00100000:1.0:1789760694.049103:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515571840 00000100:00100000:1.0:1789760694.049175:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672267008233 time 672374243660 state 0x0 ref 1 00000100:00100000:1.0:1789760694.049190:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672374504076 time 672374243660 state 0x1 ref 1 00000100:00100000:1.0:1789760694.049221:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672374504076 time 672514510168 state 0x0 ref 1 00000100:00100000:1.0:1789760694.049234:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515571840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760694.049294:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18456576, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:1.0:1789760694.049303:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.049319:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:1.0:1789760694.049324:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.049330:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.049353:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.102960:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971823 is committed 00002000:00010000:1.0:1789760694.110507:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18456576 -> 18460672 00010000:00010000:1.0:1789760694.110528:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.110540:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.110549:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.110562:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760694.110567:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.110576:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.110595:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.110601:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cda0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.110835:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515571840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61598us (61842us total) trans 4294971823 rc 0/0 00000100:00100000:1.0:1789760694.110846:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4535 00000100:00100000:1.0:1789760694.110855:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672374504076 time 672514510168 state 0x2 ref 0 00000100:00100000:0.0:1789760694.126091:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760694.126237:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515571968 00000100:00100000:1.0:1789760694.126246:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515571968 00000100:00100000:1.0:1789760694.126303:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672374504076 time 672514510168 state 0x0 ref 1 00000100:00100000:1.0:1789760694.126318:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672514770584 time 672514510168 state 0x1 ref 1 00000100:00100000:1.0:1789760694.126395:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672514770584 time 672591684508 state 0x0 ref 1 00000100:00100000:1.0:1789760694.126410:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515571968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760694.126454:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18460672, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:1.0:1789760694.126462:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.126476:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:1.0:1789760694.126480:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.126486:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.126499:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.207134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971824 is committed 00002000:00010000:0.0:1789760694.207212:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18460672 -> 18464768 00010000:00010000:0.0:1789760694.207228:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.207241:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.207250:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.207263:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760694.207268:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.207276:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.207293:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.207300:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cda7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.207431:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515571968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 81016us (81343us total) trans 4294971824 rc 0/0 00000100:00100000:0.0:1789760694.207441:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4536 00000100:00100000:0.0:1789760694.207451:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672514770584 time 672591684508 state 0x2 ref 0 00000100:00100000:0.0:1789760694.216026:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760694.216354:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515572096 00000100:00100000:1.0:1789760694.216364:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515572096 00000100:00100000:1.0:1789760694.216414:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672514770584 time 672591684508 state 0x0 ref 1 00000100:00100000:1.0:1789760694.216429:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672591944924 time 672591684508 state 0x1 ref 1 00000100:00100000:1.0:1789760694.216460:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672591944924 time 672681749165 state 0x0 ref 1 00000100:00100000:1.0:1789760694.216472:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515572096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760694.216518:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18464768, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:1.0:1789760694.216526:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.216541:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:1.0:1789760694.216545:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.216551:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.216565:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760694.285799:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18464768 -> 18468864 00010000:00010000:1.0:1789760694.285813:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.285827:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.285835:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.285843:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760694.285847:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.285855:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.285874:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.285882:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.285985:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515572096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69510us (69964us total) trans 4294971825 rc 0/0 00000100:00100000:1.0:1789760694.285995:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4537 00000100:00100000:1.0:1789760694.286004:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672591944924 time 672681749165 state 0x2 ref 0 00000100:00100000:0.0:1789760694.293918:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760694.293989:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515572224 00000100:00100000:1.0:1789760694.293998:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515572224 00000100:00100000:1.0:1789760694.294048:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672591944924 time 672681749165 state 0x0 ref 1 00000100:00100000:1.0:1789760694.294065:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672682009581 time 672681749165 state 0x1 ref 1 00000100:00100000:1.0:1789760694.294095:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672682009581 time 672759383568 state 0x0 ref 1 00000100:00100000:1.0:1789760694.294107:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515572224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760694.294154:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18468864, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:1.0:1789760694.294162:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cdb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.294177:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:1.0:1789760694.294181:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.294187:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cdb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.294207:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1cdb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18468864->18472959] (req 18468864->18472959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.336287:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760694.417173:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515572480 00000100:00100000:0.0:1789760694.417182:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515572480 00000100:00100000:0.0:1789760694.417232:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672759643984 time 672801812531 state 0x0 ref 1 00000100:00100000:0.0:1789760694.417246:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672802072947 time 672801812531 state 0x1 ref 1 00000100:00100000:0.0:1789760694.417279:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672802072947 time 672882568709 state 0x0 ref 1 00000100:00100000:0.0:1789760694.417291:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515572480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760694.417349:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18477056, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:0.0:1789760694.417358:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cdc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.417373:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760694.417377:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.417383:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cdc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.417396:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cdc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.528138:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515572608 00000100:00100000:1.0:1789760694.528149:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515572608 00000100:00100000:1.0:1789760694.528203:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672802072947 time 672882568709 state 0x0 ref 1 00000100:00100000:1.0:1789760694.528219:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672882829125 time 672882568709 state 0x1 ref 1 00000100:00100000:1.0:1789760694.528252:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672882829125 time 672993541530 state 0x0 ref 1 00000100:00100000:1.0:1789760694.528265:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515572608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760694.528313:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18481152, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:1.0:1789760694.528322:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.528354:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=672s v=(0 0) 00010000:00010000:1.0:1789760694.528359:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.528365:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.528389:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1cdca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.590859:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760694.618184:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760694.618281:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515572736 00000100:00100000:1.0:1789760694.618292:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515572736 00000100:00100000:1.0:1789760694.618353:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672882829125 time 672993541530 state 0x0 ref 1 00000100:00100000:1.0:1789760694.618368:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672993801946 time 672993541530 state 0x1 ref 1 00000100:00100000:1.0:1789760694.618399:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672993801946 time 673083687853 state 0x0 ref 1 00000100:00100000:1.0:1789760694.618411:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515572736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760694.618455:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18485248, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x9007 00010000:00010000:1.0:1789760694.618464:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.618479:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:1.0:1789760694.618483:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.618490:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.618503:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760694.759130:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18485248 -> 18489344 00010000:00010000:1.0:1789760694.759152:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.759165:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.759179:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.759186:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760694.759191:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.759200:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.759222:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.759229:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cdd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.761680:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515572736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 143260us (143495us total) trans 4294971830 rc 0/0 00000100:00100000:1.0:1789760694.761693:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4542 00000100:00100000:1.0:1789760694.761703:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 672993801946 time 673083687853 state 0x2 ref 0 00000100:00100000:0.0:1789760694.764586:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760694.919085:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971831 is committed 00002000:00010000:0.0:1789760694.919388:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18489344 -> 18493440 00010000:00010000:0.0:1789760694.919404:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.919415:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.919423:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.919431:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760694.919435:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.919443:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.919460:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.919467:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cdd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.919556:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515572864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 154682us (154975us total) trans 4294971831 rc 0/0 00000100:00100000:0.0:1789760694.919565:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4543 00000100:00100000:0.0:1789760694.919574:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673083948269 time 673230148823 state 0x2 ref 0 00000100:00100000:0.0:1789760694.923631:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760695.048110:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18493440 -> 18497536 00002000:00010000:1.0:1789760695.048119:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760694 -> 1789760695 00002000:00010000:1.0:1789760695.048122:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760694 -> 1789760695 00010000:00010000:1.0:1789760695.048136:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.048150:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.048158:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.048165:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760695.048170:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.048178:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.048195:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.048203:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cddf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.048301:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515572992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 124434us (124674us total) trans 4294971832 rc 0/0 00000100:00100000:1.0:1789760695.048311:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4544 00000100:00100000:1.0:1789760695.048321:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673230409239 time 673389142214 state 0x2 ref 0 00000001:00080000:1.0:1789760695.048372:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971832 is committed 00000100:00100000:1.0:1789760695.053398:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760695.146237:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971833 is committed 00000100:00100000:1.0:1789760695.353820:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515573504 00000100:00100000:1.0:1789760695.353830:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515573504 00000100:00100000:1.0:1789760695.353885:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673619859477 time 673702885513 state 0x0 ref 1 00000100:00100000:1.0:1789760695.353900:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673703145929 time 673702885513 state 0x1 ref 1 00000100:00100000:1.0:1789760695.353930:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673703145929 time 673819218679 state 0x0 ref 1 00000100:00100000:1.0:1789760695.353941:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515573504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760695.353990:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18509824, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x9007 00010000:00010000:1.0:1789760695.353999:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cdfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.354014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:1.0:1789760695.354019:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.354025:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cdfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.354039:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1cdfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.446744:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515573632 00000100:00100000:0.0:1789760695.446754:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515573632 00000100:00100000:0.0:1789760695.446801:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673703145929 time 673819218679 state 0x0 ref 1 00000100:00100000:0.0:1789760695.446815:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673819479095 time 673819218679 state 0x1 ref 1 00000100:00100000:0.0:1789760695.446842:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673819479095 time 673912131187 state 0x0 ref 1 00000100:00100000:0.0:1789760695.446853:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515573632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760695.446894:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18513920, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x9007 00010000:00010000:0.0:1789760695.446902:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.446917:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760695.446922:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.447085:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.447103:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760695.533462:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18513920 -> 18518016 00010000:00010000:1.0:1789760695.533479:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.533492:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.533502:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.533510:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760695.533515:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.533524:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.533544:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.533552:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ce02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.533643:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515573632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 86785us (87074us total) trans 4294971837 rc 0/0 00000100:00100000:1.0:1789760695.533653:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4549 00000100:00100000:1.0:1789760695.533662:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673819479095 time 673912131187 state 0x2 ref 0 00000100:00100000:1.0:1789760695.539282:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515573760 00000100:00100000:1.0:1789760695.539292:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515573760 00000100:00100000:1.0:1789760695.539352:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673819479095 time 673912131187 state 0x0 ref 1 00000100:00100000:1.0:1789760695.539367:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673912391603 time 673912131187 state 0x1 ref 1 00000100:00100000:1.0:1789760695.539396:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673912391603 time 674004685613 state 0x0 ref 1 00000100:00100000:1.0:1789760695.539408:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515573760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760695.539451:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18518016, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x9007 00010000:00010000:1.0:1789760695.539460:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.539475:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760695.539479:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.539485:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.539501:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760695.602768:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18518016 -> 18522112 00010000:00010000:1.0:1789760695.602783:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.602796:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.602809:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.602816:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760695.602821:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.602830:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.602848:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.602856:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.602942:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515573760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63530us (64004us total) trans 4294971838 rc 0/0 00000100:00100000:1.0:1789760695.602952:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4550 00000100:00100000:1.0:1789760695.602962:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 673912391603 time 674004685613 state 0x2 ref 0 00010000:00010000:0.0:1789760695.619104:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760695.640259:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971839 is committed 00000100:00100000:0.0:1789760695.649052:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515574016 00000100:00100000:0.0:1789760695.649063:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515574016 00000100:00100000:0.0:1789760695.649114:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674004946029 time 674072429016 state 0x0 ref 1 00000100:00100000:0.0:1789760695.649130:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674072689432 time 674072429016 state 0x1 ref 1 00000100:00100000:0.0:1789760695.649161:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674072689432 time 674114450342 state 0x0 ref 1 00000100:00100000:0.0:1789760695.649174:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515574016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760695.649220:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18526208, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x9007 00010000:00010000:0.0:1789760695.649229:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ce17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.649248:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.649253:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.649259:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ce17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.649272:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ce17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.800913:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515574272 00000100:00100000:1.0:1789760695.800923:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515574272 00000100:00100000:1.0:1789760695.800984:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674114710758 time 674193300497 state 0x0 ref 1 00000100:00100000:1.0:1789760695.800999:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674193560913 time 674193300497 state 0x1 ref 1 00000100:00100000:1.0:1789760695.801033:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674193560913 time 674266322888 state 0x0 ref 1 00000100:00100000:1.0:1789760695.801054:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515574272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760695.801104:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18534400, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x9007 00010000:00010000:1.0:1789760695.801113:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.801128:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760695.801132:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.801138:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.801151:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ce25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.806657:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544402688:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.806734:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760695.806773:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544403200:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.806797:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760695.806820:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760695.806827:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544403072:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.806850:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760695.806875:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544402944:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.806895:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760695.806958:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1789760695.806964:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544402816:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.807461:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760695.807480:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be32d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544403328:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.807499:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760695.807546:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403200 00000100:00100000:1.0:1789760695.807554:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403200 00000100:00100000:1.0:1789760695.807596:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544402944 00000100:00100000:1.0:1789760695.807600:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544402944 00000100:00100000:1.0:1789760695.807619:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544402816 00000100:00100000:1.0:1789760695.807622:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544402816 00000100:00100000:1.0:1789760695.807637:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1789760695.807645:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544403200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.807695:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544403200:12345-0@lo:400:kworker.0 Request processed in 51us (897us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760695.807703:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1789760695.807739:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789760695.807745:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544402944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.807780:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544402944:12345-0@lo:400:kworker.0 Request processed in 35us (883us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760695.807786:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789760695.807796:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1789760695.807802:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544402816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.807828:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544402816:12345-0@lo:400:kworker.0 Request processed in 27us (367us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760695.807835:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1789760695.807852:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403072 00000100:00100000:1.0:1789760695.807856:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403072 00000100:00100000:1.0:1789760695.807876:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403328 00000100:00100000:1.0:1789760695.807879:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403328 00000100:00100000:1.0:1789760695.807897:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:1.0:1789760695.807903:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544403072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.807932:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544403072:12345-0@lo:400:kworker.0 Request processed in 30us (1081us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760695.807938:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:1.0:1789760695.807947:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:1.0:1789760695.807953:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544403328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.810531:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544403328:12345-0@lo:400:kworker.0 Request processed in 2570us (3023us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760695.810544:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:1.0:1789760695.810622:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544403072:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.810640:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544403200:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.810682:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544402816:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.810695:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be32d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544403328:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.810723:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544402944:0@lo:400:kworker.0 00000100:00100000:1.0:1789760695.819143:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544402688:0@lo:400:kworker.0 00000001:00080000:0.0:1789760695.869167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971842 is committed 00000001:00080000:0.0:1789760695.973522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971843 is committed 00000100:00100000:1.0:1789760695.976078:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515574528 00000100:00100000:1.0:1789760695.976088:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515574528 00000100:00100000:1.0:1789760695.976136:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674266583304 time 674347375463 state 0x0 ref 1 00000100:00100000:1.0:1789760695.976150:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674347635879 time 674347375463 state 0x1 ref 1 00000100:00100000:1.0:1789760695.976178:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674347635879 time 674441467338 state 0x0 ref 1 00000100:00100000:1.0:1789760695.976189:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515574528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760695.976249:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18542592, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x9007 00010000:00010000:1.0:1789760695.976258:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ce33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.976272:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:1.0:1789760695.976277:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.976283:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ce33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.976296:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ce33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.037186:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971844 is committed 00000100:00100000:0.0:1789760696.037765:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00080000:1.0:1789760696.038474:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760696.091145:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760696.091295:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515574784 00000100:00100000:0.0:1789760696.091305:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515574784 00000100:00100000:0.0:1789760696.091480:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674441727754 time 674503317481 state 0x0 ref 1 00000100:00100000:0.0:1789760696.091496:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674503577897 time 674503317481 state 0x1 ref 1 00000100:00100000:0.0:1789760696.091527:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 674503577897 time 674556816166 state 0x0 ref 1 00000100:00100000:0.0:1789760696.091541:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515574784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760696.091587:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18550784, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x9007 00010000:00010000:0.0:1789760696.091597:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ce41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.091617:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760696.091622:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.091628:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ce41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.091642:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ce41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.242233:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760696.242279:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760696.242311:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760696.246673:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515575040 00000100:00100000:1.0:1789760696.246687:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515575040 00000100:00100000:1.0:1789760696.246794:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 411 00000100:00100000:1.0:1789760696.246808:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515575040:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760696.246888:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515575040:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 86us (4665us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760696.246896:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 411 00000100:00100000:1.0:1789760696.246911:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515575168 00000100:00100000:1.0:1789760696.246916:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515575168 00000100:00100000:1.0:1789760696.246954:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 412 00000100:00100000:1.0:1789760696.246961:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515575168:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760696.246988:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515575168:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 27us (4676us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760696.246994:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 412 00000100:00080000:0.0:1789760696.247586:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760696.266833:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760696.337194:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760696.458911:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760696.555333:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971849 is committed 00000100:00100000:0.0:1789760696.565360:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760696.638070:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760696.639541:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971850 is committed 00000100:00100000:0.0:1789760696.650609:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760696.721042:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971851 is committed 00002000:00010000:0.0:1789760696.721126:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18571264 -> 18575360 00010000:00010000:0.0:1789760696.721142:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.721154:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.721167:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.721174:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760696.721179:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.721187:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.721205:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.721213:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ce64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.721695:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515575808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70857us (71089us total) trans 4294971851 rc 0/0 00000100:00100000:0.0:1789760696.721744:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4563 00000100:00100000:0.0:1789760696.721756:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675032434966 time 675116108744 state 0x2 ref 0 00000100:00100000:0.0:1789760696.724957:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760696.814307:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971852 is committed 00002000:00010000:1.0:1789760696.814528:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18575360 -> 18579456 00010000:00010000:1.0:1789760696.814543:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.814556:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.814564:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.814572:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760696.814576:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.814585:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.814604:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.814617:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ce6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.814749:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515575936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 87549us (89754us total) trans 4294971852 rc 0/0 00000100:00100000:1.0:1789760696.814759:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4564 00000100:00100000:1.0:1789760696.814769:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675116369160 time 675192431680 state 0x2 ref 0 00002000:00010000:0.0:1789760696.881452:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18579456 -> 18583552 00002000:00010000:0.0:1789760696.881457:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760696 -> 1789760697 00002000:00010000:0.0:1789760696.881460:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760696 -> 1789760697 00010000:00010000:0.0:1789760696.881472:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.881485:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.881496:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.881504:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760696.881508:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.881517:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.881536:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.881545:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ce72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.881636:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515576064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61943us (62264us total) trans 4294971853 rc 0/0 00000100:00100000:0.0:1789760696.881646:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4565 00000100:00100000:0.0:1789760696.881655:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675192692096 time 675284966841 state 0x2 ref 0 00000100:00100000:0.0:1789760696.889801:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760697.041377:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971855 is committed 00000100:00100000:0.0:1789760697.049108:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760697.130804:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760697.201443:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515576704 00000100:00100000:1.0:1789760697.201453:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515576704 00000100:00100000:1.0:1789760697.201506:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675514909066 time 675596334709 state 0x0 ref 1 00000100:00100000:1.0:1789760697.201521:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675596595125 time 675596334709 state 0x1 ref 1 00000100:00100000:1.0:1789760697.201551:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675596595125 time 675666839973 state 0x0 ref 1 00000100:00100000:1.0:1789760697.201564:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515576704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760697.201613:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18599936, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x9007 00010000:00010000:1.0:1789760697.201632:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.201649:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:1.0:1789760697.201653:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.201660:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.201675:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760697.262335:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18599936 -> 18604032 00010000:00010000:1.0:1789760697.262355:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.262367:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.262377:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.262385:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760697.262389:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.262397:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.262416:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.262424:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ce95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.262512:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515576704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60946us (61165us total) trans 4294971858 rc 0/0 00000100:00100000:1.0:1789760697.262522:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4570 00000100:00100000:1.0:1789760697.262532:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675596595125 time 675666839973 state 0x2 ref 0 00000100:00100000:0.0:1789760697.266893:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515576832 00000100:00100000:0.0:1789760697.266903:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515576832 00000100:00100000:0.0:1789760697.266955:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675596595125 time 675666839973 state 0x0 ref 1 00000100:00100000:0.0:1789760697.266970:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675667100389 time 675666839973 state 0x1 ref 1 00000100:00100000:0.0:1789760697.267000:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675667100389 time 675732289413 state 0x0 ref 1 00000100:00100000:0.0:1789760697.267013:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515576832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760697.267073:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18604032, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x9007 00010000:00010000:0.0:1789760697.267083:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.267098:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760697.267102:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.267108:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.267122:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760697.338215:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18604032 -> 18608128 00010000:00010000:0.0:1789760697.338238:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338252:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338262:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338270:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760697.338275:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338284:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338303:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.338320:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ce9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.338521:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515576832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71503us (71896us total) trans 4294971859 rc 0/0 00000100:00100000:0.0:1789760697.338532:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4571 00000100:00100000:0.0:1789760697.338543:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675667100389 time 675732289413 state 0x2 ref 0 00000100:00100000:0.0:1789760697.355891:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760697.386828:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971860 is committed 00000001:00080000:1.0:1789760697.471610:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971861 is committed 00000100:00100000:0.0:1789760697.485905:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515577216 00000100:00100000:0.0:1789760697.485917:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515577216 00000100:00100000:0.0:1789760697.485985:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675823299485 time 675856824207 state 0x0 ref 1 00000100:00100000:0.0:1789760697.486000:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675857084623 time 675856824207 state 0x1 ref 1 00000100:00100000:0.0:1789760697.486038:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675857084623 time 675951327174 state 0x0 ref 1 00000100:00100000:0.0:1789760697.486051:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515577216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760697.486097:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18616320, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x9007 00010000:00010000:0.0:1789760697.486105:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ceb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.486120:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760697.486125:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.486133:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ceb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.486146:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ceb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.531507:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:2.0:1789760697.532223:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=675s v=(0 0) 00010000:00010000:2.0:1789760697.532228:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760697.532240:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.532255:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760697.593824:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18620416 -> 18624512 00010000:00010000:0.0:1789760697.593838:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.593852:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.593862:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.593870:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760697.593874:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.593884:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.593904:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.593912:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ceb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.594003:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515577344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62075us (62500us total) trans 4294971863 rc 0/0 00000100:00100000:0.0:1789760697.594013:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4575 00000100:00100000:0.0:1789760697.594023:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675951587590 time 675997203276 state 0x2 ref 0 00000100:00100000:1.0:1789760697.604318:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515577472 00000100:00100000:1.0:1789760697.604328:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515577472 00000100:00100000:1.0:1789760697.604378:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675951587590 time 675997203276 state 0x0 ref 1 00000100:00100000:1.0:1789760697.604392:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675997463692 time 675997203276 state 0x1 ref 1 00000100:00100000:1.0:1789760697.604450:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675997463692 time 676069739811 state 0x0 ref 1 00000100:00100000:1.0:1789760697.604462:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515577472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760697.604508:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18624512, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x9007 00010000:00010000:1.0:1789760697.604517:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1cebf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.604533:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760697.604537:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.604542:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1cebf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.604563:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1cebf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.663977:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760697.723257:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971864 is committed 00000100:00100000:2.0:1789760697.729445:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515577600 00000100:00100000:2.0:1789760697.729455:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515577600 00000100:00100000:2.0:1789760697.729507:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 675997463692 time 676069739811 state 0x0 ref 1 00000100:00100000:2.0:1789760697.729522:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676070000227 time 676069739811 state 0x1 ref 1 00000100:00100000:2.0:1789760697.729661:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676070000227 time 676194948301 state 0x0 ref 1 00000100:00100000:2.0:1789760697.729673:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515577600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760697.729756:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18628608, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x9007 00010000:00010000:2.0:1789760697.729776:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1cec6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.729793:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760697.729798:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760697.729808:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1cec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.729823:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1cec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.785553:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971865 is committed 00000100:00100000:0.0:1789760697.796336:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760697.822546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971866 is committed 00000100:00100000:0.0:1789760697.835107:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760697.853812:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760697.853832:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80540bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544403456:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760697.853893:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760697.853948:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403456 00000100:00100000:2.0:1789760697.853954:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403456 00000100:00100000:2.0:1789760697.853997:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789760697.854005:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544403456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760697.854146:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760697.854189:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544403456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 184us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760697.854196:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789760697.854295:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80540bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544403456:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1789760697.854414:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760697.854450:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760697.854459:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8010e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544403584:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760697.854486:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760697.854910:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403584 00000100:00100000:0.0:1789760697.854915:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403584 00000100:00100000:0.0:1789760697.854947:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:0.0:1789760697.854954:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544403584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760697.854967:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3839885312 avail 3834642432 00000100:00100000:0.0:1789760697.855000:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544403584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (513us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760697.855006:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00080000:0.0:1789760697.855024:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760697.855445:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8010e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544403584:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760697.878270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971867 is committed 00000100:00100000:1.0:1789760697.882283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760697.882378:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515577984 00000100:00100000:0.0:1789760697.882387:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515577984 00000100:00100000:0.0:1789760697.882435:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676262119939 time 676302324688 state 0x0 ref 1 00000100:00100000:0.0:1789760697.882449:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676302585104 time 676302324688 state 0x1 ref 1 00000100:00100000:0.0:1789760697.882475:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676302585104 time 676347764842 state 0x0 ref 1 00000100:00100000:0.0:1789760697.882487:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515577984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760697.882533:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18640896, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:0.0:1789760697.882542:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.882558:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760697.882563:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.882569:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.882582:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760697.912885:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18640896 -> 18644992 00010000:00010000:1.0:1789760697.912901:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.912915:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.912924:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.912931:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760697.912936:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.912946:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.912965:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.912979:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cedb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.913066:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515577984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30576us (30787us total) trans 4294971868 rc 0/0 00000100:00100000:1.0:1789760697.913076:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4580 00000100:00100000:1.0:1789760697.913085:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676302585104 time 676347764842 state 0x2 ref 0 00000100:00100000:0.0:1789760697.916407:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760697.916701:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515578112 00000100:00100000:2.0:1789760697.916936:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515578112 00000001:00080000:0.0:1789760697.960029:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971869 is committed 00000100:00100000:0.0:1789760697.964303:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760697.964404:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515578240 00000100:00100000:1.0:1789760697.964415:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515578240 00000100:00100000:1.0:1789760697.964464:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676348025258 time 676382803476 state 0x0 ref 1 00000100:00100000:1.0:1789760697.964479:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676383063892 time 676382803476 state 0x1 ref 1 00000100:00100000:1.0:1789760697.964506:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676383063892 time 676429794341 state 0x0 ref 1 00000100:00100000:1.0:1789760697.964518:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515578240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760697.964563:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18649088, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:1.0:1789760697.964572:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cee9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.964589:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760697.964592:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.964599:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cee9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.964616:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1cee9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.002945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971870 is committed 00000100:00100000:1.0:1789760698.015138:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00080000:0.0:1789760698.015849:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760698.069737:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971871 is committed 00000100:00100000:0.0:1789760698.073128:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760698.102596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971872 is committed 00000100:00100000:0.0:1789760698.108742:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760698.151551:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971873 is committed 00000100:00100000:1.0:1789760698.155886:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760698.156145:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515578752 00000100:00100000:0.0:1789760698.156156:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515578752 00000100:00100000:0.0:1789760698.156210:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676538857518 time 676574498379 state 0x0 ref 1 00000100:00100000:0.0:1789760698.156226:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676574758795 time 676574498379 state 0x1 ref 1 00000100:00100000:0.0:1789760698.156262:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676574758795 time 676621550658 state 0x0 ref 1 00000100:00100000:0.0:1789760698.156274:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515578752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760698.156330:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18665472, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:0.0:1789760698.156339:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.156355:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760698.156369:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.156376:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.156389:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.195649:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18665472 -> 18669568 00010000:00010000:0.0:1789760698.195670:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.195684:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.195692:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.195699:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.195727:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.195738:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.195827:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.195835:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.196218:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515578752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39940us (40335us total) trans 4294971874 rc 0/0 00000100:00100000:0.0:1789760698.196236:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4586 00000100:00100000:0.0:1789760698.196247:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676574758795 time 676621550658 state 0x2 ref 0 00000001:00080000:1.0:1789760698.196382:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971874 is committed 00000100:00100000:1.0:1789760698.200610:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515578880 00000100:00100000:1.0:1789760698.200620:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515578880 00000100:00100000:1.0:1789760698.200670:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676574758795 time 676621550658 state 0x0 ref 1 00000100:00100000:1.0:1789760698.200684:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676621811074 time 676621550658 state 0x1 ref 1 00000100:00100000:1.0:1789760698.200732:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676621811074 time 676666020735 state 0x0 ref 1 00000100:00100000:1.0:1789760698.200753:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515578880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760698.200800:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18669568, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:1.0:1789760698.200809:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.200824:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:1.0:1789760698.200830:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.200835:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.200849:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.238626:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18669568 -> 18673664 00010000:00010000:0.0:1789760698.238640:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.238654:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.238663:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.238671:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.238676:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.238702:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.238753:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.238761:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1cf0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.238834:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515578880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38086us (38294us total) trans 4294971875 rc 0/0 00000100:00100000:0.0:1789760698.238844:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4587 00000100:00100000:0.0:1789760698.238853:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676621811074 time 676666020735 state 0x2 ref 0 00000100:00100000:0.0:1789760698.243780:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760698.243914:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515579008 00000100:00100000:2.0:1789760698.243924:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515579008 00000100:00100000:2.0:1789760698.243986:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676621811074 time 676666020735 state 0x0 ref 1 00000100:00100000:2.0:1789760698.244000:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676666281151 time 676666020735 state 0x1 ref 1 00000100:00100000:2.0:1789760698.244038:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676666281151 time 676709319987 state 0x0 ref 1 00000100:00100000:2.0:1789760698.244050:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515579008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760698.244094:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18673664, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:2.0:1789760698.244103:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.244118:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760698.244122:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.244127:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.244140:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.286378:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18673664 -> 18677760 00010000:00010000:0.0:1789760698.286393:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.286406:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.286417:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.286430:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.286436:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.286446:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.286465:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.286473:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cf13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.286553:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515579008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42501us (42778us total) trans 4294971876 rc 0/0 00000100:00100000:0.0:1789760698.286563:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4588 00000100:00100000:0.0:1789760698.286573:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676666281151 time 676709319987 state 0x2 ref 0 00000100:00100000:1.0:1789760698.292106:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760698.335768:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971877 is committed 00002000:00010000:0.0:1789760698.335910:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18677760 -> 18681856 00010000:00010000:0.0:1789760698.335924:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335937:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335952:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335959:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.335963:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335971:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335990:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335998:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1cf1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.336086:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515579136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43742us (43983us total) trans 4294971877 rc 0/0 00000100:00100000:0.0:1789760698.336095:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4589 00000100:00100000:0.0:1789760698.336104:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676709580403 time 676757618432 state 0x2 ref 0 00000100:00100000:2.0:1789760698.349700:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515579264 00000100:00100000:2.0:1789760698.349742:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515579264 00000100:00100000:2.0:1789760698.349802:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676709580403 time 676757618432 state 0x0 ref 1 00000100:00100000:2.0:1789760698.349816:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676757878848 time 676757618432 state 0x1 ref 1 00000100:00100000:2.0:1789760698.349850:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676757878848 time 676815138832 state 0x0 ref 1 00000100:00100000:2.0:1789760698.349861:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515579264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760698.349908:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18681856, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:2.0:1789760698.349916:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.349932:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760698.350084:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.350091:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.350114:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.397834:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971878 is committed 00002000:00010000:2.0:1789760698.397937:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18681856 -> 18685952 00010000:00010000:2.0:1789760698.398023:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.398037:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.398046:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.398053:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760698.398058:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.398067:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.398086:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.398094:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cf21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.398194:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515579264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48330us (48591us total) trans 4294971878 rc 0/0 00000100:00100000:2.0:1789760698.398205:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4590 00000100:00100000:2.0:1789760698.398214:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676757878848 time 676815138832 state 0x2 ref 0 00000100:00100000:0.0:1789760698.401194:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515579392 00000100:00100000:0.0:1789760698.401205:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515579392 00000100:00100000:0.0:1789760698.401267:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676757878848 time 676815138832 state 0x0 ref 1 00000100:00100000:0.0:1789760698.401285:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676815399248 time 676815138832 state 0x1 ref 1 00000100:00100000:0.0:1789760698.401325:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676815399248 time 676866614545 state 0x0 ref 1 00000100:00100000:0.0:1789760698.401337:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+6:2362:x1876699515579392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760698.401383:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18685952, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:0.0:1789760698.401391:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.401405:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760698.401409:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.401416:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.401429:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cf28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.433811:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971879 is committed 00000100:00100000:1.0:1789760698.438197:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760698.438321:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515579520 00000100:00100000:0.0:1789760698.438330:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515579520 00000100:00100000:0.0:1789760698.438383:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676815399248 time 676866614545 state 0x0 ref 1 00000100:00100000:0.0:1789760698.438399:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676866874961 time 676866614545 state 0x1 ref 1 00000100:00100000:0.0:1789760698.438461:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676866874961 time 676903750339 state 0x0 ref 1 00000100:00100000:0.0:1789760698.438474:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515579520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760698.438518:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18690048, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:0.0:1789760698.438527:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.438543:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760698.438548:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.438554:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.438568:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.485539:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971880 is committed 00002000:00010000:0.0:1789760698.485880:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18690048 -> 18694144 00010000:00010000:0.0:1789760698.485894:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.485908:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.485916:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.485922:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.485926:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.485935:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.485965:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.485972:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.486065:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515579520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47589us (47873us total) trans 4294971880 rc 0/0 00000100:00100000:0.0:1789760698.486075:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4592 00000100:00100000:0.0:1789760698.486084:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676866874961 time 676903750339 state 0x2 ref 0 00000100:00100000:0.0:1789760698.490131:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760698.490211:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515579648 00000100:00100000:2.0:1789760698.490220:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515579648 00000100:00100000:2.0:1789760698.490336:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676866874961 time 676903750339 state 0x0 ref 1 00000100:00100000:2.0:1789760698.490352:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676904010755 time 676903750339 state 0x1 ref 1 00000100:00100000:2.0:1789760698.490448:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676904010755 time 676955736786 state 0x0 ref 1 00000100:00100000:2.0:1789760698.490460:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515579648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760698.490510:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18694144, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:2.0:1789760698.490519:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490535:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=676s v=(0 0) 00010000:00010000:2.0:1789760698.490540:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.490545:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.490559:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.541764:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971881 is committed 00002000:00010000:2.0:1789760698.541939:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18694144 -> 18698240 00010000:00010000:2.0:1789760698.542026:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.542040:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.542050:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.542058:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760698.542063:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.542073:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.542096:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.542105:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.542243:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515579648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51779us (52115us total) trans 4294971881 rc 0/0 00000100:00100000:2.0:1789760698.542258:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4593 00000100:00100000:2.0:1789760698.542269:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676904010755 time 676955736786 state 0x2 ref 0 00000100:00100000:2.0:1789760698.551056:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760698.555256:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676955997202 time 677020542907 state 0x0 ref 1 00000100:00100000:2.0:1789760698.555278:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515579776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760698.555340:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18698240, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:2.0:1789760698.555349:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.555375:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760698.555379:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.555385:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.555398:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.616966:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971882 is committed 00002000:00010000:1.0:1789760698.617129:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18698240 -> 18702336 00010000:00010000:1.0:1789760698.617143:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.617156:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.617165:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.617173:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760698.617177:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.617185:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.617204:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.617211:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cf3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.617326:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515579776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62053us (66275us total) trans 4294971882 rc 0/0 00000100:00100000:1.0:1789760698.617337:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4594 00000100:00100000:1.0:1789760698.617347:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 676955997202 time 677020542907 state 0x2 ref 0 00000100:00100000:0.0:1789760698.631685:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760698.687149:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760698.726801:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18702336 -> 18706432 00010000:00010000:0.0:1789760698.726819:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.726832:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.726841:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.726848:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.726854:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.726863:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.726882:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.726890:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1cf44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.726986:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515579904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 95040us (95306us total) trans 4294971883 rc 0/0 00000100:00100000:0.0:1789760698.726995:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4595 00000100:00100000:0.0:1789760698.727005:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677020803323 time 677097221077 state 0x2 ref 0 00000001:00080000:2.0:1789760698.727547:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971883 is committed 00000100:00100000:2.0:1789760698.731941:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760698.766794:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971884 is committed 00002000:00010000:0.0:1789760698.766891:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18706432 -> 18710528 00010000:00010000:0.0:1789760698.766909:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.766923:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.766933:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.766957:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.766962:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.766971:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.766991:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.766998:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1cf4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.767095:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515580032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34680us (35158us total) trans 4294971884 rc 0/0 00000100:00100000:0.0:1789760698.767105:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4596 00000100:00100000:0.0:1789760698.767116:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677097481493 time 677197532126 state 0x2 ref 0 00000100:00100000:1.0:1789760698.772412:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760698.772493:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515580160 00000100:00100000:0.0:1789760698.772504:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515580160 00000100:00100000:0.0:1789760698.772559:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677097481493 time 677197532126 state 0x0 ref 1 00000100:00100000:0.0:1789760698.772574:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677197792542 time 677197532126 state 0x1 ref 1 00000100:00100000:0.0:1789760698.772604:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677197792542 time 677237892836 state 0x0 ref 1 00000100:00100000:0.0:1789760698.772619:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515580160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760698.772667:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18710528, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x9007 00010000:00010000:0.0:1789760698.772676:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.772693:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.772697:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.772703:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.772753:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760698.817112:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18710528 -> 18714624 00010000:00010000:2.0:1789760698.817129:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.817149:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.817158:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.817166:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760698.817171:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.817180:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.817201:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.817210:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cf52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.817546:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515580160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44922us (45198us total) trans 4294971885 rc 0/0 00000100:00100000:2.0:1789760698.817558:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4597 00000100:00100000:2.0:1789760698.817569:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677197792542 time 677237892836 state 0x2 ref 0 00000100:00100000:2.0:1789760698.821811:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760698.903121:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971886 is committed 00002000:00010000:0.0:1789760698.903475:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18714624 -> 18718720 00002000:00010000:0.0:1789760698.903480:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760698 -> 1789760699 00002000:00010000:0.0:1789760698.903483:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760698 -> 1789760699 00010000:00010000:0.0:1789760698.903497:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.903510:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.903519:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.903531:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.903536:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.903544:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.903562:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.903570:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1cf59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.903702:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515580288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 81699us (81895us total) trans 4294971886 rc 0/0 00000100:00100000:0.0:1789760698.903752:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4598 00000100:00100000:0.0:1789760698.903762:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677238153252 time 677287274594 state 0x2 ref 0 00000100:00100000:1.0:1789760698.909811:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760698.909899:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515580416 00000100:00100000:2.0:1789760698.909909:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515580416 00000100:00100000:2.0:1789760698.909959:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677238153252 time 677287274594 state 0x0 ref 1 00000100:00100000:2.0:1789760698.909975:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677287535010 time 677287274594 state 0x1 ref 1 00000100:00100000:2.0:1789760698.910000:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677287535010 time 677375289477 state 0x0 ref 1 00000100:00100000:2.0:1789760698.910012:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515580416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760698.910066:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18718720, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:2.0:1789760698.910075:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.910089:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760698.910093:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.910099:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.910111:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.984694:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971887 is committed 00002000:00010000:0.0:1789760698.984870:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18718720 -> 18722816 00010000:00010000:0.0:1789760698.984886:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.984899:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.984913:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.984920:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760698.984924:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.984933:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.984953:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.984961:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1cf60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.985067:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515580416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75052us (75260us total) trans 4294971887 rc 0/0 00000100:00100000:0.0:1789760698.985076:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4599 00000100:00100000:0.0:1789760698.985087:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677287535010 time 677375289477 state 0x2 ref 0 00000100:00100000:1.0:1789760699.001537:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760699.001634:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515580544 00000100:00100000:2.0:1789760699.001643:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515580544 00000100:00100000:2.0:1789760699.001694:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677287535010 time 677375289477 state 0x0 ref 1 00000100:00100000:2.0:1789760699.001748:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677375549893 time 677375289477 state 0x1 ref 1 00000100:00100000:2.0:1789760699.001773:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677375549893 time 677467063144 state 0x0 ref 1 00000100:00100000:2.0:1789760699.001786:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515580544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760699.001831:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18722816, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:2.0:1789760699.001840:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.001856:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760699.001860:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.001867:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.001880:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.078922:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971888 is committed 00002000:00010000:0.0:1789760699.086544:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18722816 -> 18726912 00010000:00010000:0.0:1789760699.086567:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.086585:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.086593:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.086609:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760699.086615:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.086623:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.086643:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.086650:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.086829:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515580544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 85041us (85297us total) trans 4294971888 rc 0/0 00000100:00100000:0.0:1789760699.086840:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4600 00000100:00100000:0.0:1789760699.086850:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677375549893 time 677467063144 state 0x2 ref 0 00000100:00100000:1.0:1789760699.092034:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760699.153546:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18726912 -> 18731008 00010000:00010000:1.0:1789760699.153563:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.153575:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.153583:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.153590:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760699.153595:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.153604:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.153623:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.153631:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1cf6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.153769:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515580672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61547us (61739us total) trans 4294971889 rc 0/0 00000100:00100000:1.0:1789760699.153780:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4601 00000100:00100000:1.0:1789760699.153790:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677467323560 time 677557496709 state 0x2 ref 0 00000100:00100000:1.0:1789760699.158939:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760699.159016:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515580800 00000100:00100000:0.0:1789760699.159036:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515580800 00000100:00100000:0.0:1789760699.159092:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677467323560 time 677557496709 state 0x0 ref 1 00000100:00100000:0.0:1789760699.159107:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677557757125 time 677557496709 state 0x1 ref 1 00000100:00100000:0.0:1789760699.159140:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677557757125 time 677624429698 state 0x0 ref 1 00000100:00100000:0.0:1789760699.159151:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515580800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760699.159195:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18731008, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:0.0:1789760699.159203:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cf75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.159218:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760699.159223:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.159230:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cf75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.159243:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1cf75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.247555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971890 is committed 00000100:00100000:0.0:1789760699.251584:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760699.251911:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515580928 00000100:00100000:1.0:1789760699.251921:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515580928 00000100:00100000:1.0:1789760699.251970:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677557757125 time 677624429698 state 0x0 ref 1 00000100:00100000:1.0:1789760699.251986:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677624690114 time 677624429698 state 0x1 ref 1 00000100:00100000:1.0:1789760699.252011:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677624690114 time 677717300615 state 0x0 ref 1 00000100:00100000:1.0:1789760699.252033:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515580928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760699.252078:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18735104, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:1.0:1789760699.252087:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cf7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.252109:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:1.0:1789760699.252113:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.252120:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cf7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.252134:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1cf7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.313467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971891 is committed 00000100:00100000:0.0:1789760699.321650:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760699.389628:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971892 is committed 00000100:00100000:0.0:1789760699.393418:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760699.393512:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515581184 00000100:00100000:2.0:1789760699.393521:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515581184 00000100:00100000:2.0:1789760699.393574:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677717561031 time 677787140517 state 0x0 ref 1 00000100:00100000:2.0:1789760699.393589:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677787400933 time 677787140517 state 0x1 ref 1 00000100:00100000:2.0:1789760699.393649:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677787400933 time 677858937892 state 0x0 ref 1 00000100:00100000:2.0:1789760699.393661:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515581184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760699.393740:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18743296, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:2.0:1789760699.393748:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cf8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.393763:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760699.393768:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.393773:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cf8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.393785:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1cf8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.423579:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971893 is committed 00000100:00100000:0.0:1789760699.428116:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760699.428224:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515581312 00000100:00100000:2.0:1789760699.428234:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515581312 00000100:00100000:2.0:1789760699.428284:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677787400933 time 677858937892 state 0x0 ref 1 00000100:00100000:2.0:1789760699.428308:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677859198308 time 677858937892 state 0x1 ref 1 00000100:00100000:2.0:1789760699.428336:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677859198308 time 677893624983 state 0x0 ref 1 00000100:00100000:2.0:1789760699.428348:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515581312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760699.428397:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18747392, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:2.0:1789760699.428406:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.428422:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=677s v=(0 0) 00010000:00010000:2.0:1789760699.428427:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.428433:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.428447:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.475036:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971894 is committed 00002000:00010000:1.0:1789760699.475224:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18747392 -> 18751488 00010000:00010000:1.0:1789760699.475240:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.475253:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.475262:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.475269:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760699.475273:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.475283:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.475315:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.475323:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1cf91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.475417:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515581312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47064us (47303us total) trans 4294971894 rc 0/0 00000100:00100000:1.0:1789760699.475426:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4606 00000100:00100000:1.0:1789760699.475436:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677859198308 time 677893624983 state 0x2 ref 0 00000100:00100000:1.0:1789760699.479567:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760699.546688:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971895 is committed 00002000:00010000:2.0:1789760699.546774:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18751488 -> 18755584 00010000:00010000:2.0:1789760699.546789:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.546800:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.546808:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.546814:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760699.546819:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.546828:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.546848:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.546860:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1cf98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.546962:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515581440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67137us (67397us total) trans 4294971895 rc 0/0 00000100:00100000:2.0:1789760699.546973:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4607 00000100:00100000:2.0:1789760699.546982:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677893885399 time 677945097450 state 0x2 ref 0 00000100:00100000:2.0:1789760699.550764:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:2.0:1789760699.621702:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18755584 -> 18759680 00010000:00010000:2.0:1789760699.621755:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.621766:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.621779:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.621786:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760699.621791:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.621800:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.621824:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.621832:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1cf9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.621912:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515581568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70937us (71154us total) trans 4294971896 rc 0/0 00000100:00100000:2.0:1789760699.621922:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4608 00000100:00100000:2.0:1789760699.621931:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 677945357866 time 678016247901 state 0x2 ref 0 00000001:00080000:2.0:1789760699.621967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971896 is committed 00000100:00100000:0.0:1789760699.626538:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760699.713544:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760699.727611:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971897 is committed 00002000:00010000:0.0:1789760699.732092:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18759680 -> 18763776 00010000:00010000:0.0:1789760699.732113:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.732125:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.732134:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.732141:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760699.732146:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.732154:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.732173:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.732180:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1cfa6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.732508:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515581696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 105710us (105974us total) trans 4294971897 rc 0/0 00000100:00100000:0.0:1789760699.732519:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4609 00000100:00100000:0.0:1789760699.732529:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678016508317 time 678092073171 state 0x2 ref 0 00000100:00100000:2.0:1789760699.742603:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760699.742685:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515581824 00000100:00100000:0.0:1789760699.742695:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515581824 00000100:00100000:0.0:1789760699.742773:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678016508317 time 678092073171 state 0x0 ref 1 00000100:00100000:0.0:1789760699.742788:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678092333587 time 678092073171 state 0x1 ref 1 00000100:00100000:0.0:1789760699.742815:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678092333587 time 678208105057 state 0x0 ref 1 00000100:00100000:0.0:1789760699.742826:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515581824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760699.742872:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18763776, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:0.0:1789760699.742880:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.742895:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.742899:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.742905:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.742917:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.824139:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971898 is committed 00002000:00010000:0.0:1789760699.824533:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18763776 -> 18767872 00010000:00010000:0.0:1789760699.824549:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.824560:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.824570:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.824578:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760699.824583:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.824592:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.824612:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.824619:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1cfad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.824755:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515581824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 81882us (82125us total) trans 4294971898 rc 0/0 00000100:00100000:0.0:1789760699.824765:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4610 00000100:00100000:0.0:1789760699.824774:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678092333587 time 678208105057 state 0x2 ref 0 00000100:00100000:2.0:1789760699.831165:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515581952 00000100:00100000:2.0:1789760699.831175:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515581952 00000100:00100000:2.0:1789760699.831241:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678092333587 time 678208105057 state 0x0 ref 1 00000100:00100000:2.0:1789760699.831255:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678208365473 time 678208105057 state 0x1 ref 1 00000100:00100000:2.0:1789760699.831286:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678208365473 time 678296575016 state 0x0 ref 1 00000100:00100000:2.0:1789760699.831308:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515581952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760699.831356:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18767872, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x9007 00010000:00010000:2.0:1789760699.831365:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.831379:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760699.831384:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.831390:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.831405:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.877904:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18767872 -> 18771968 00002000:00010000:2.0:1789760699.877912:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760699 -> 1789760700 00002000:00010000:2.0:1789760699.877915:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760699 -> 1789760700 00010000:00010000:2.0:1789760699.877928:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.877952:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.877962:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.877970:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760699.877975:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.877990:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.878010:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.878018:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1cfb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.878112:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515581952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46802us (47044us total) trans 4294971899 rc 0/0 00000100:00100000:2.0:1789760699.878123:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4611 00000100:00100000:2.0:1789760699.878133:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678208365473 time 678296575016 state 0x2 ref 0 00000001:00080000:1.0:1789760699.878668:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971899 is committed 00000100:00100000:1.0:1789760699.881850:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760699.881938:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515582080 00000100:00100000:2.0:1789760699.882012:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515582080 00000100:00100000:2.0:1789760699.882066:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678208365473 time 678296575016 state 0x0 ref 1 00000100:00100000:2.0:1789760699.882081:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678296835432 time 678296575016 state 0x1 ref 1 00000100:00100000:2.0:1789760699.882146:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678296835432 time 678347433920 state 0x0 ref 1 00000100:00100000:2.0:1789760699.882158:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515582080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760699.882205:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18771968, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760699.882214:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cfbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.882306:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760699.882311:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.882318:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cfbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.882332:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1cfbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.919173:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971900 is committed 00000100:00100000:0.0:1789760699.923354:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760699.957529:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971901 is committed 00000100:00100000:0.0:1789760699.962400:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760699.993481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971902 is committed 00000100:00100000:0.0:1789760699.994104:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515582336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31453us (31706us total) trans 4294971902 rc 0/0 00000100:00100000:0.0:1789760699.994115:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4614 00000100:00100000:0.0:1789760699.994124:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678389617001 time 678427924058 state 0x2 ref 0 00000100:00100000:1.0:1789760700.001233:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760700.001530:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515582464 00000100:00100000:2.0:1789760700.001540:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515582464 00000100:00080000:1.0:1789760700.001610:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760700.001612:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678389617001 time 678427924058 state 0x0 ref 1 00000100:00100000:2.0:1789760700.001629:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678428184474 time 678427924058 state 0x1 ref 1 00000100:00100000:2.0:1789760700.001659:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678428184474 time 678466948244 state 0x0 ref 1 00000100:00100000:2.0:1789760700.001672:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515582464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760700.001745:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18784256, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760700.001754:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cfd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001770:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760700.001774:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.001781:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cfd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.001800:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1cfd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.041425:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971903 is committed 00000100:00100000:2.0:1789760700.045538:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760700.045679:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515582592 00000100:00100000:0.0:1789760700.045688:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515582592 00000100:00100000:0.0:1789760700.045761:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678428184474 time 678466948244 state 0x0 ref 1 00000100:00100000:0.0:1789760700.045775:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678467208660 time 678466948244 state 0x1 ref 1 00000100:00100000:0.0:1789760700.045796:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678467208660 time 678511086340 state 0x0 ref 1 00000100:00100000:0.0:1789760700.045807:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515582592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760700.045851:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18788352, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:0.0:1789760700.045859:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.045873:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760700.045878:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.045884:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.045897:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760700.079173:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18788352 -> 18792448 00010000:00010000:2.0:1789760700.079192:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.079205:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.079227:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.079234:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760700.079239:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.079249:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.079269:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.079278:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1cfd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.079377:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515582592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33566us (33855us total) trans 4294971904 rc 0/0 00000100:00100000:2.0:1789760700.079388:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4616 00000100:00100000:2.0:1789760700.079397:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678467208660 time 678511086340 state 0x2 ref 0 00000100:00100000:0.0:1789760700.083394:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760700.083466:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515582720 00000100:00100000:2.0:1789760700.083477:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515582720 00000100:00100000:2.0:1789760700.083597:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678467208660 time 678511086340 state 0x0 ref 1 00000100:00100000:2.0:1789760700.083612:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678511346756 time 678511086340 state 0x1 ref 1 00000100:00100000:2.0:1789760700.083642:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678511346756 time 678548930853 state 0x0 ref 1 00000100:00100000:2.0:1789760700.083654:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515582720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760700.083701:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18792448, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760700.083810:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cfde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.083825:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760700.083829:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.083836:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cfde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.083849:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1cfde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18792448->18796543] (req 18792448->18796543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.122403:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760700.236103:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18796544 -> 18800640 00010000:00010000:0.0:1789760700.236120:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.236133:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.236142:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.236150:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760700.236154:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.236164:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.236183:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.236440:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1cfe5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18796544->18800639] (req 18796544->18800639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.236568:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515582848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 113503us (114168us total) trans 4294971906 rc 0/0 00000100:00100000:0.0:1789760700.236579:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4618 00000100:00100000:0.0:1789760700.236589:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678549191269 time 678588337182 state 0x2 ref 0 00000100:00100000:2.0:1789760700.247247:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760700.281785:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971907 is committed 00002000:00010000:2.0:1789760700.281882:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18800640 -> 18804736 00010000:00010000:2.0:1789760700.281897:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.281910:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.281920:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.281928:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760700.281943:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.281953:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.281974:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.281982:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1cfec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18800640->18804735] (req 18800640->18804735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.282108:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515582976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34570us (34865us total) trans 4294971907 rc 0/0 00000100:00100000:2.0:1789760700.282119:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4619 00000100:00100000:2.0:1789760700.282129:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678588597598 time 678712803644 state 0x2 ref 0 00000100:00100000:2.0:1789760700.287174:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760700.287751:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515583104 00000100:00100000:0.0:1789760700.287763:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515583104 00000100:00100000:0.0:1789760700.287816:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678588597598 time 678712803644 state 0x0 ref 1 00000100:00100000:0.0:1789760700.287827:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678713064060 time 678712803644 state 0x1 ref 1 00000100:00100000:0.0:1789760700.287878:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678713064060 time 678753167788 state 0x0 ref 1 00000100:00100000:0.0:1789760700.287887:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515583104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760700.287929:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18804736, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:0.0:1789760700.287992:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.288005:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760700.288008:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.288013:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.288027:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760700.323547:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18804736 -> 18808832 00010000:00010000:0.0:1789760700.323566:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.323579:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.323588:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.323595:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760700.323600:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.323609:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.323627:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.323633:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1cff3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.323758:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515583104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35868us (36588us total) trans 4294971908 rc 0/0 00000100:00100000:0.0:1789760700.323772:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4620 00000100:00100000:0.0:1789760700.323781:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678713064060 time 678753167788 state 0x2 ref 0 00000001:00080000:1.0:1789760700.324596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971908 is committed 00000100:00100000:0.0:1789760700.335776:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760700.335866:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515583232 00000100:00100000:2.0:1789760700.335877:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515583232 00000100:00100000:2.0:1789760700.335932:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678713064060 time 678753167788 state 0x0 ref 1 00000100:00100000:2.0:1789760700.335947:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678753428204 time 678753167788 state 0x1 ref 1 00000100:00100000:2.0:1789760700.335975:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678753428204 time 678801264636 state 0x0 ref 1 00000100:00100000:2.0:1789760700.335987:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515583232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760700.336036:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18808832, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760700.336045:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1cffa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.336061:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760700.336065:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.336072:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1cffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.336085:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1cffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.365773:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971909 is committed 00000100:00100000:2.0:1789760700.373930:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515583360 00000100:00100000:2.0:1789760700.373941:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515583360 00000100:00100000:2.0:1789760700.373992:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678753428204 time 678801264636 state 0x0 ref 1 00000100:00100000:2.0:1789760700.374006:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678801525052 time 678801264636 state 0x1 ref 1 00000100:00100000:2.0:1789760700.374075:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678801525052 time 678839362985 state 0x0 ref 1 00000100:00100000:2.0:1789760700.374087:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515583360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760700.374137:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18812928, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760700.374146:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.374163:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760700.374167:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.374173:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.374187:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760700.403219:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18812928 -> 18817024 00010000:00010000:2.0:1789760700.403239:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.403252:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.403267:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.403274:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760700.403279:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.403287:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.403490:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.403499:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d001 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18812928->18817023] (req 18812928->18817023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.403598:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515583360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29508us (30096us total) trans 4294971910 rc 0/0 00000100:00100000:2.0:1789760700.403608:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4622 00000100:00100000:2.0:1789760700.403618:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678801525052 time 678839362985 state 0x2 ref 0 00000001:00080000:1.0:1789760700.407097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971910 is committed 00000100:00100000:0.0:1789760700.407698:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760700.441318:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971911 is committed 00002000:00010000:2.0:1789760700.441488:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18817024 -> 18821120 00010000:00010000:2.0:1789760700.441504:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.441530:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.441540:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.441548:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760700.441553:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.441562:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.441581:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.441588:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d008 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.441678:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515583488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33586us (33981us total) trans 4294971911 rc 0/0 00000100:00100000:2.0:1789760700.441688:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4623 00000100:00100000:2.0:1789760700.441697:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678839623401 time 678873364576 state 0x2 ref 0 00000100:00100000:1.0:1789760700.445233:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760700.445687:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515583616 00000100:00100000:2.0:1789760700.445698:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515583616 00000100:00100000:2.0:1789760700.445840:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678839623401 time 678873364576 state 0x0 ref 1 00000100:00100000:2.0:1789760700.445855:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678873624992 time 678873364576 state 0x1 ref 1 00000100:00100000:2.0:1789760700.445919:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678873624992 time 678911208155 state 0x0 ref 1 00000100:00100000:2.0:1789760700.445941:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515583616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760700.445994:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18821120, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760700.446002:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.446018:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=678s v=(0 0) 00010000:00010000:2.0:1789760700.446022:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.446028:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.446042:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.480128:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971912 is committed 00002000:00010000:0.0:1789760700.480247:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18821120 -> 18825216 00010000:00010000:0.0:1789760700.480262:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.480275:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.480285:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.480368:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760700.480374:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.480384:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.480406:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.480414:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d00f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.480527:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515583616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34583us (35290us total) trans 4294971912 rc 0/0 00000100:00100000:0.0:1789760700.480539:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4624 00000100:00100000:0.0:1789760700.480550:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678873624992 time 678911208155 state 0x2 ref 0 00000100:00100000:2.0:1789760700.486180:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515583744 00000100:00100000:2.0:1789760700.486193:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515583744 00000001:00080000:1.0:1789760700.548779:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971913 is committed 00000100:00100000:0.0:1789760700.558864:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760700.616185:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971914 is committed 00000100:00100000:1.0:1789760700.621865:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760700.622424:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 678952197927 time 679024340496 state 0x0 ref 1 00000100:00100000:0.0:1789760700.622440:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679024600912 time 679024340496 state 0x1 ref 1 00000100:00100000:0.0:1789760700.622524:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679024600912 time 679087804786 state 0x0 ref 1 00000100:00100000:0.0:1789760700.622538:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515584000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760700.622583:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18833408, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:0.0:1789760700.622593:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.622609:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.622613:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.622619:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.622631:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760700.659811:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971915 is committed 00002000:00010000:0.0:1789760700.659912:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18833408 -> 18837504 00010000:00010000:0.0:1789760700.659936:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.659949:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.659959:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.659967:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760700.659972:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.659981:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.660002:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.660010:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d024 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.660101:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515584000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37560us (38239us total) trans 4294971915 rc 0/0 00000100:00100000:0.0:1789760700.660114:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4627 00000100:00100000:0.0:1789760700.660124:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679024600912 time 679087804786 state 0x2 ref 0 00000100:00100000:2.0:1789760700.664322:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760700.664400:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515584128 00000100:00100000:0.0:1789760700.664410:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515584128 00000100:00100000:0.0:1789760700.664466:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679024600912 time 679087804786 state 0x0 ref 1 00000100:00100000:0.0:1789760700.664482:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679088065202 time 679087804786 state 0x1 ref 1 00000100:00100000:0.0:1789760700.664510:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679088065202 time 679129799911 state 0x0 ref 1 00000100:00100000:0.0:1789760700.664530:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515584128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760700.664576:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18837504, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:0.0:1789760700.664585:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.664600:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.664605:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.664610:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.664624:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760700.711829:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18837504 -> 18841600 00010000:00010000:2.0:1789760700.711849:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.711867:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.711876:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.711884:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760700.711889:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.711898:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.711918:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.711925:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d02b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.712123:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515584128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47589us (47804us total) trans 4294971916 rc 0/0 00000100:00100000:2.0:1789760700.712133:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4628 00000100:00100000:2.0:1789760700.712143:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679088065202 time 679129799911 state 0x2 ref 0 00000001:00080000:1.0:1789760700.712303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971916 is committed 00000100:00100000:1.0:1789760700.716432:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760700.716498:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515584256 00000100:00100000:2.0:1789760700.716508:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515584256 00000100:00100000:2.0:1789760700.716568:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679088065202 time 679129799911 state 0x0 ref 1 00000100:00100000:2.0:1789760700.716582:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679130060327 time 679129799911 state 0x1 ref 1 00000100:00100000:2.0:1789760700.716610:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679130060327 time 679181899254 state 0x0 ref 1 00000100:00100000:2.0:1789760700.716621:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515584256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760700.716667:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18841600, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:2.0:1789760700.716676:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.716690:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789760700.716695:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.716701:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.716733:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.733812:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760700.759282:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18841600 -> 18845696 00010000:00010000:0.0:1789760700.759313:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.759326:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.759335:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.759343:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760700.759348:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.759358:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.759377:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.759385:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d032 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.759478:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515584256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42854us (43051us total) trans 4294971917 rc 0/0 00000100:00100000:0.0:1789760700.759488:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4629 00000100:00100000:0.0:1789760700.759499:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679130060327 time 679181899254 state 0x2 ref 0 00000001:00080000:1.0:1789760700.759918:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971917 is committed 00000100:00100000:1.0:1789760700.763997:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760700.764070:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515584384 00000100:00100000:0.0:1789760700.764079:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515584384 00000100:00100000:0.0:1789760700.764132:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679130060327 time 679181899254 state 0x0 ref 1 00000100:00100000:0.0:1789760700.764148:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679182159670 time 679181899254 state 0x1 ref 1 00000100:00100000:0.0:1789760700.764178:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679182159670 time 679229467113 state 0x0 ref 1 00000100:00100000:0.0:1789760700.764189:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515584384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760700.764249:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18845696, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x9007 00010000:00010000:0.0:1789760700.764257:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.764273:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.764277:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.764283:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.764311:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.796334:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18845696 -> 18849792 00010000:00010000:1.0:1789760700.796354:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.796369:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.796379:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.796387:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760700.796391:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.796400:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.796420:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.796428:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d039 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.796531:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515584384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32338us (32538us total) trans 4294971918 rc 0/0 00000100:00100000:1.0:1789760700.796546:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4630 00000100:00100000:1.0:1789760700.796555:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679182159670 time 679229467113 state 0x2 ref 0 00000100:00100000:2.0:1789760700.802615:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760700.847526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971919 is committed 00000100:00100000:2.0:1789760700.851910:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760700.904704:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971920 is committed 00000100:00100000:0.0:1789760700.909113:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760700.929592:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c61500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544403712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.929626:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760700.929658:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c62680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544404224:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.929685:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760700.929702:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760700.929724:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c60a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544404096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.929749:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760700.930090:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c61500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544403712:0@lo:400:kworker.0 00000100:00100000:0.0:1789760700.930199:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403968 00000100:00100000:0.0:1789760700.930212:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403968 00000100:00100000:0.0:1789760700.930274:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544403840 00000100:00100000:0.0:1789760700.930278:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544403840 00000100:00100000:2.0:1789760700.930356:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404224 00000100:00100000:2.0:1789760700.930361:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404224 00000100:00100000:2.0:1789760700.930393:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789760700.930400:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544403968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.930441:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544403968:12345-0@lo:400:kworker.0 Request processed in 41us (1329us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760700.930447:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789760700.930460:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:2.0:1789760700.930466:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544403840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.930493:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544403840:12345-0@lo:400:kworker.0 Request processed in 27us (1293us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760700.930498:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:2.0:1789760700.930509:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789760700.930515:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544404224:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.930545:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544404224:12345-0@lo:400:kworker.0 Request processed in 30us (859us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760700.930551:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789760700.930576:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c61c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544403840:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.930585:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c60380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544403968:0@lo:400:kworker.0 00000100:00100000:2.0:1789760700.930611:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c62680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544404224:0@lo:400:kworker.0 00000100:00080000:1.0:1789760700.930674:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760700.930679:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760700.930688:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404096 00000100:00100000:1.0:1789760700.930692:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404096 00000100:00100000:1.0:1789760700.930745:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1789760700.930752:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544404096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760700.930791:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544404096:12345-0@lo:400:kworker.0 Request processed in 39us (1040us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760700.930797:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1789760700.930827:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c60a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544404096:0@lo:400:kworker.0 00000100:00080000:2.0:1789760700.930945:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760700.951498:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971921 is committed 00000100:00100000:2.0:1789760700.954096:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760700.954191:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515584896 00000100:00100000:0.0:1789760700.954200:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515584896 00000100:00100000:0.0:1789760700.954266:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679317747207 time 679376540464 state 0x0 ref 1 00000100:00100000:0.0:1789760700.954283:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679376800880 time 679376540464 state 0x1 ref 1 00000100:00100000:0.0:1789760700.954322:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679376800880 time 679419611268 state 0x0 ref 1 00000100:00100000:0.0:1789760700.954335:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515584896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760700.954381:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18862080, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9007 00010000:00010000:0.0:1789760700.954391:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d055 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954405:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.954410:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.954416:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954429:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.023065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971922 is committed 00000100:00100000:0.0:1789760701.032152:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760701.032472:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515585024 00000100:00100000:2.0:1789760701.032482:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515585024 00000100:00100000:2.0:1789760701.032534:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679376800880 time 679419611268 state 0x0 ref 1 00000100:00100000:2.0:1789760701.032549:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679419871684 time 679419611268 state 0x1 ref 1 00000100:00100000:2.0:1789760701.032583:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679419871684 time 679497872141 state 0x0 ref 1 00000100:00100000:2.0:1789760701.032594:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515585024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760701.032641:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18866176, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9007 00010000:00010000:2.0:1789760701.032650:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d05c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.032667:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789760701.032672:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.032679:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d05c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.032692:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d05c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.120959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971923 is committed 00000100:00100000:2.0:1789760701.128658:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760701.128753:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515585152 00000100:00100000:1.0:1789760701.128763:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515585152 00000100:00100000:1.0:1789760701.128813:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679419871684 time 679497872141 state 0x0 ref 1 00000100:00100000:1.0:1789760701.128828:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679498132557 time 679497872141 state 0x1 ref 1 00000100:00100000:1.0:1789760701.128859:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679498132557 time 679594146495 state 0x0 ref 1 00000100:00100000:1.0:1789760701.128872:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515585152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760701.128927:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18870272, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9007 00010000:00010000:1.0:1789760701.128937:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d063 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.128958:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:1.0:1789760701.128963:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.128969:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.128982:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.202052:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971924 is committed 00000100:00100000:2.0:1789760701.210610:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760701.265670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971925 is committed 00000100:00100000:0.0:1789760701.271300:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760701.313872:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971926 is committed 00000100:00100000:2.0:1789760701.319790:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760701.358425:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18882560 -> 18886656 00010000:00010000:0.0:1789760701.358447:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.358467:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.358476:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.358483:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760701.358488:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.358497:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.358516:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.358523:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d078 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.358619:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515585536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38453us (38831us total) trans 4294971927 rc 0/0 00000100:00100000:0.0:1789760701.358629:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4639 00000100:00100000:0.0:1789760701.358638:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679737194717 time 679785438868 state 0x2 ref 0 00000001:00080000:2.0:1789760701.358732:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971927 is committed 00000100:00100000:2.0:1789760701.362922:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760701.363103:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760701.363136:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760701.363168:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515585920 00000100:00100000:2.0:1789760701.363177:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515585920 00000100:00100000:2.0:1789760701.363241:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515585792 00000100:00100000:2.0:1789760701.363244:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515585792 00000100:00100000:2.0:1789760701.363269:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 416 00000100:00100000:2.0:1789760701.363278:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515585920:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760701.363360:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515585920:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 83us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760701.363369:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 416 00000100:00100000:2.0:1789760701.363385:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 417 00000100:00100000:2.0:1789760701.363392:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806191c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515585792:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760701.363416:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806191c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515585792:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 25us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760701.363423:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 417 00000100:00080000:2.0:1789760701.363438:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760701.365620:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515586048 00000100:00100000:0.0:1789760701.365625:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515586048 00000100:00100000:0.0:1789760701.365662:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679737194717 time 679785438868 state 0x0 ref 1 00000100:00100000:0.0:1789760701.365676:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679785699284 time 679785438868 state 0x1 ref 1 00000100:00100000:0.0:1789760701.365702:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679785699284 time 679830991137 state 0x0 ref 1 00000100:00100000:0.0:1789760701.365742:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515586048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760701.365782:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18886656, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9807 00010000:00010000:0.0:1789760701.365800:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.365815:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760701.365820:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.365826:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.365840:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760701.399166:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18886656 -> 18890752 00010000:00010000:0.0:1789760701.399188:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.399210:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.399227:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.399235:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760701.399239:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.399248:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.399268:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.399275:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d07f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.399372:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515586048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33627us (34065us total) trans 4294971928 rc 0/0 00000100:00100000:0.0:1789760701.399382:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4640 00000100:00100000:0.0:1789760701.399391:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679785699284 time 679830991137 state 0x2 ref 0 00000100:00100000:1.0:1789760701.403851:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760701.455215:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760701.512274:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971930 is committed 00000100:00100000:1.0:1789760701.525544:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760701.585419:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971931 is committed 00000100:00100000:0.0:1789760701.590650:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515586560 00000100:00100000:0.0:1789760701.590660:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515586560 00000100:00100000:0.0:1789760701.590766:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679921542808 time 679991332235 state 0x0 ref 1 00000100:00100000:0.0:1789760701.590782:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679991592651 time 679991332235 state 0x1 ref 1 00000100:00100000:0.0:1789760701.590820:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679991592651 time 680056108624 state 0x0 ref 1 00000100:00100000:0.0:1789760701.590833:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515586560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760701.590884:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18903040, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9807 00010000:00010000:0.0:1789760701.590894:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d09b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.590908:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760701.590912:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.590918:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d09b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.590943:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d09b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.642576:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971932 is committed 00000100:00100000:2.0:1789760701.643382:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515586560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52546us (52830us total) trans 4294971932 rc 0/0 00000100:00100000:2.0:1789760701.643393:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4644 00000100:00100000:2.0:1789760701.643409:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679991592651 time 680056108624 state 0x2 ref 0 00000100:00100000:2.0:1789760701.649542:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760701.649638:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515586688 00000100:00100000:0.0:1789760701.649648:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515586688 00000100:00100000:0.0:1789760701.649702:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 679991592651 time 680056108624 state 0x0 ref 1 00000100:00100000:0.0:1789760701.649778:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680056369040 time 680056108624 state 0x1 ref 1 00000100:00100000:0.0:1789760701.650023:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680056369040 time 680115310481 state 0x0 ref 1 00000100:00100000:0.0:1789760701.650037:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515586688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760701.650087:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18907136, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9807 00010000:00010000:0.0:1789760701.650098:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d0a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.650115:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760701.650120:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.650127:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d0a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.650142:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d0a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.700867:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971933 is committed 00000100:00100000:1.0:1789760701.704631:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760701.704735:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515586816 00000100:00100000:0.0:1789760701.704744:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515586816 00000100:00100000:0.0:1789760701.704877:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680056369040 time 680115310481 state 0x0 ref 1 00000100:00100000:0.0:1789760701.704893:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680115570897 time 680115310481 state 0x1 ref 1 00000100:00100000:0.0:1789760701.705016:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680115570897 time 680170212274 state 0x0 ref 1 00000100:00100000:0.0:1789760701.705032:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515586816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760701.705078:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18911232, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x9807 00010000:00010000:0.0:1789760701.705087:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.705103:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760701.705107:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.705113:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.705127:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.738303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971934 is committed 00002000:00010000:1.0:1789760701.739030:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18911232 -> 18915328 00010000:00010000:1.0:1789760701.739050:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.739063:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.739073:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.739081:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760701.739086:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.739095:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.739116:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.739124:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d0a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.739473:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515586816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34438us (34843us total) trans 4294971934 rc 0/0 00000100:00100000:1.0:1789760701.739483:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4646 00000100:00100000:1.0:1789760701.739492:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680115570897 time 680170212274 state 0x2 ref 0 00000100:00100000:1.0:1789760701.745856:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760701.757784:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760701.807684:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971935 is committed 00002000:00010000:2.0:1789760701.808052:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18915328 -> 18919424 00010000:00010000:2.0:1789760701.808068:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.808081:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.808091:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.808103:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760701.808108:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.808116:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.808142:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.808150:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d0b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.808363:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515586944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62249us (62509us total) trans 4294971935 rc 0/0 00000100:00100000:2.0:1789760701.808376:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4647 00000100:00100000:2.0:1789760701.808386:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680170472690 time 680211388177 state 0x2 ref 0 00000100:00100000:1.0:1789760701.814560:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760701.873598:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971936 is committed 00000100:00100000:2.0:1789760701.876549:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760701.909665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971937 is committed 00002000:00010000:1.0:1789760701.909778:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18923520 -> 18927616 00010000:00010000:1.0:1789760701.909795:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.909808:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.909819:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.909832:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760701.909837:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.909846:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.909867:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.909874:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d0be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.909978:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806193800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515587200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33103us (33434us total) trans 4294971937 rc 0/0 00000100:00100000:1.0:1789760701.909991:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4649 00000100:00100000:1.0:1789760701.910000:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680280508901 time 680342146339 state 0x2 ref 0 00000100:00100000:2.0:1789760701.914982:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760701.966166:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971938 is committed 00002000:00010000:1.0:1789760701.966431:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18927616 -> 18931712 00010000:00010000:1.0:1789760701.966450:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.966463:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.966478:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.966486:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760701.966491:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.966501:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.966523:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.966532:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.966659:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515587328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51421us (51681us total) trans 4294971938 rc 0/0 00000100:00100000:1.0:1789760701.966670:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4650 00000100:00100000:1.0:1789760701.966680:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680342406755 time 680380512267 state 0x2 ref 0 00000100:00100000:2.0:1789760701.970272:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760701.970436:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515587456 00000100:00100000:0.0:1789760701.970446:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515587456 00000100:00100000:0.0:1789760701.970499:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680342406755 time 680380512267 state 0x0 ref 1 00000100:00100000:0.0:1789760701.970515:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680380772683 time 680380512267 state 0x1 ref 1 00000100:00100000:0.0:1789760701.970580:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680380772683 time 680435868150 state 0x0 ref 1 00000100:00100000:0.0:1789760701.970594:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515587456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760701.970646:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18931712, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:0.0:1789760701.970655:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.970674:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760701.970678:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.970685:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.970698:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.034913:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971939 is committed 00002000:00010000:2.0:1789760702.035016:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18931712 -> 18935808 00010000:00010000:2.0:1789760702.035031:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.035044:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.035053:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.035067:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760702.035071:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.035091:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.035111:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.035118:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d0cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.035219:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515587456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64623us (64951us total) trans 4294971939 rc 0/0 00000100:00100000:2.0:1789760702.035228:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4651 00000100:00100000:2.0:1789760702.035237:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680380772683 time 680435868150 state 0x2 ref 0 00000100:00100000:2.0:1789760702.041196:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.041418:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515587584 00000100:00100000:1.0:1789760702.041428:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515587584 00000100:00100000:1.0:1789760702.041500:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680380772683 time 680435868150 state 0x0 ref 1 00000100:00100000:1.0:1789760702.041516:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680436128566 time 680435868150 state 0x1 ref 1 00000100:00100000:1.0:1789760702.041549:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680436128566 time 680506836904 state 0x0 ref 1 00000100:00100000:1.0:1789760702.041562:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515587584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.041612:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18935808, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:1.0:1789760702.041621:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041637:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760702.041642:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.041650:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.041665:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760702.045652:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760702.088429:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971940 is committed 00002000:00010000:1.0:1789760702.088546:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18935808 -> 18939904 00010000:00010000:1.0:1789760702.088562:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088576:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088585:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088592:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760702.088597:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088606:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088624:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.088633:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.088757:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515587584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47192us (47565us total) trans 4294971940 rc 0/0 00000100:00100000:1.0:1789760702.088768:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4652 00000100:00100000:1.0:1789760702.088779:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680436128566 time 680506836904 state 0x2 ref 0 00000100:00100000:0.0:1789760702.097110:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760702.097457:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515587712 00000100:00100000:2.0:1789760702.097468:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515587712 00000100:00100000:2.0:1789760702.097522:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680436128566 time 680506836904 state 0x0 ref 1 00000100:00100000:2.0:1789760702.097536:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680507097320 time 680506836904 state 0x1 ref 1 00000100:00100000:2.0:1789760702.097570:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680507097320 time 680562859282 state 0x0 ref 1 00000100:00100000:2.0:1789760702.097582:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515587712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760702.097628:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18939904, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:2.0:1789760702.097637:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.097652:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760702.097656:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.097662:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.097677:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.146491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971941 is committed 00002000:00010000:0.0:1789760702.146602:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18939904 -> 18944000 00010000:00010000:0.0:1789760702.146617:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.146631:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.146639:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.146647:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760702.146651:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.146661:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.146680:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.146688:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d0da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.146887:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515587712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49302us (49781us total) trans 4294971941 rc 0/0 00000100:00100000:0.0:1789760702.146898:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4653 00000100:00100000:0.0:1789760702.146908:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680507097320 time 680562859282 state 0x2 ref 0 00000100:00100000:2.0:1789760702.150319:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760702.153539:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680563119698 time 680618824610 state 0x0 ref 1 00000100:00100000:2.0:1789760702.153557:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515587840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760702.153607:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18944000, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:2.0:1789760702.153616:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.153631:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:2.0:1789760702.153635:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.153642:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.153655:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.183677:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971942 is committed 00002000:00010000:2.0:1789760702.183801:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18944000 -> 18948096 00010000:00010000:2.0:1789760702.183818:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.183831:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.183840:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.183848:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760702.183853:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.183862:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.183880:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.183889:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d0e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.183986:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515587840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30429us (33671us total) trans 4294971942 rc 0/0 00000100:00100000:2.0:1789760702.183997:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4654 00000100:00100000:2.0:1789760702.184006:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680563119698 time 680618824610 state 0x2 ref 0 00000100:00100000:2.0:1789760702.189574:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760702.189919:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18948096, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:0.0:1789760702.189930:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.189946:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760702.189950:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.189957:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.189972:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760702.221551:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18948096 -> 18952192 00010000:00010000:0.0:1789760702.221572:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.221586:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.221601:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.221609:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760702.221614:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.221622:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.221641:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.221648:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d0e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.221857:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515587968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32046us (32288us total) trans 4294971943 rc 0/0 00000100:00100000:0.0:1789760702.221868:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4655 00000100:00100000:0.0:1789760702.221878:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680619085026 time 680655084138 state 0x2 ref 0 00000001:00080000:0.0:1789760702.221915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971943 is committed 00000100:00100000:0.0:1789760702.226260:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.226525:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515588096 00000100:00100000:1.0:1789760702.226535:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515588096 00000100:00100000:1.0:1789760702.226588:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680619085026 time 680655084138 state 0x0 ref 1 00000100:00100000:1.0:1789760702.226603:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680655344554 time 680655084138 state 0x1 ref 1 00000100:00100000:1.0:1789760702.226662:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680655344554 time 680691950648 state 0x0 ref 1 00000100:00100000:1.0:1789760702.226674:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515588096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.226758:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18952192, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:1.0:1789760702.226767:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.226783:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760702.226788:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.226794:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.226808:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d0ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.271441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971944 is committed 00000100:00100000:0.0:1789760702.276149:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515588224 00000100:00100000:0.0:1789760702.276160:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515588224 00000100:00100000:0.0:1789760702.276228:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680655344554 time 680691950648 state 0x0 ref 1 00000100:00100000:0.0:1789760702.276244:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680692211064 time 680691950648 state 0x1 ref 1 00000100:00100000:0.0:1789760702.276288:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680692211064 time 680741564547 state 0x0 ref 1 00000100:00100000:0.0:1789760702.276301:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515588224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760702.276361:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18956288, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:0.0:1789760702.276370:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.276385:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760702.276389:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.276395:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.276409:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.310086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971945 is committed 00002000:00010000:1.0:1789760702.310189:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18956288 -> 18960384 00010000:00010000:1.0:1789760702.310213:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.310227:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.310238:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.310245:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760702.310250:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.310259:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.310291:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.310299:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d0f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.310386:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515588224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34082us (34401us total) trans 4294971945 rc 0/0 00000100:00100000:1.0:1789760702.310396:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4657 00000100:00100000:1.0:1789760702.310406:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680692211064 time 680741564547 state 0x2 ref 0 00000100:00100000:0.0:1789760702.314569:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.314649:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515588352 00000100:00100000:1.0:1789760702.314659:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515588352 00000100:00100000:1.0:1789760702.314754:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680692211064 time 680741564547 state 0x0 ref 1 00000100:00100000:1.0:1789760702.314769:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680741824963 time 680741564547 state 0x1 ref 1 00000100:00100000:1.0:1789760702.314788:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680741824963 time 680780077765 state 0x0 ref 1 00000100:00100000:1.0:1789760702.314799:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515588352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.314847:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18960384, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:1.0:1789760702.314857:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.314873:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760702.314879:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.314884:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.314898:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.350494:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18960384 -> 18964480 00010000:00010000:1.0:1789760702.350515:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350528:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350537:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350543:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760702.350548:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350557:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350576:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.350589:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d0fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.350680:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515588352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35877us (36116us total) trans 4294971946 rc 0/0 00000100:00100000:1.0:1789760702.350690:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4658 00000100:00100000:1.0:1789760702.350701:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680741824963 time 680780077765 state 0x2 ref 0 00000001:00080000:0.0:1789760702.351060:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971946 is committed 00000100:00100000:0.0:1789760702.354506:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.354660:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515588480 00000100:00100000:1.0:1789760702.354670:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515588480 00000100:00100000:1.0:1789760702.354761:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680741824963 time 680780077765 state 0x0 ref 1 00000100:00100000:1.0:1789760702.354776:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680780338181 time 680780077765 state 0x1 ref 1 00000100:00100000:1.0:1789760702.354805:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680780338181 time 680820094872 state 0x0 ref 1 00000100:00100000:1.0:1789760702.354818:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515588480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.354872:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18964480, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:1.0:1789760702.354881:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.354896:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760702.354900:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.354907:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.354939:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.409666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971947 is committed 00002000:00010000:0.0:1789760702.409774:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18964480 -> 18968576 00010000:00010000:0.0:1789760702.409790:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.409821:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.409831:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.409838:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760702.409843:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.409852:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.409871:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.409878:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d104 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.410015:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515588480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55194us (55514us total) trans 4294971947 rc 0/0 00000100:00100000:0.0:1789760702.410026:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4659 00000100:00100000:0.0:1789760702.410036:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680780338181 time 680820094872 state 0x2 ref 0 00000100:00100000:2.0:1789760702.414119:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760702.464681:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18968576 -> 18972672 00010000:00010000:0.0:1789760702.464698:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.464738:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.464748:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.464755:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760702.464759:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.464768:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.464788:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.464796:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d10b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.464878:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515588608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50260us (50766us total) trans 4294971948 rc 0/0 00000100:00100000:0.0:1789760702.464888:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4660 00000100:00100000:0.0:1789760702.464897:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680820355288 time 680879890067 state 0x2 ref 0 00000100:00100000:2.0:1789760702.471256:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.472324:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515588736 00000100:00100000:1.0:1789760702.472336:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515588736 00000100:00100000:1.0:1789760702.472395:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680820355288 time 680879890067 state 0x0 ref 1 00000100:00100000:1.0:1789760702.472411:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680880150483 time 680879890067 state 0x1 ref 1 00000100:00100000:1.0:1789760702.472435:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680880150483 time 680937724349 state 0x0 ref 1 00000100:00100000:1.0:1789760702.472447:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515588736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.472496:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18972672, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:1.0:1789760702.472505:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.472521:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=680s v=(0 0) 00010000:00010000:1.0:1789760702.472526:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.472533:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.472546:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760702.533479:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18972672 -> 18976768 00010000:00010000:2.0:1789760702.533496:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.533508:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.533517:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.533524:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760702.533529:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.533538:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.533562:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.533570:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d112 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.533681:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515588736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61232us (62430us total) trans 4294971949 rc 0/0 00000100:00100000:2.0:1789760702.533692:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4661 00000100:00100000:2.0:1789760702.533704:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680880150483 time 680937724349 state 0x2 ref 0 00000100:00100000:0.0:1789760702.604527:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.606765:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515588992 00000100:00100000:1.0:1789760702.606781:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515588992 00000100:00100000:0.0:1789760702.607013:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 680937984765 time 681006046913 state 0x0 ref 1 00000100:00100000:0.0:1789760702.607032:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681006307329 time 681006046913 state 0x1 ref 1 00000100:00100000:0.0:1789760702.607111:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681006307329 time 681072399125 state 0x0 ref 1 00000100:00100000:0.0:1789760702.607126:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515588992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760702.607185:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18980864, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:0.0:1789760702.607194:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.607220:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.607225:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.607231:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.607245:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760702.646837:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18980864 -> 18984960 00010000:00010000:0.0:1789760702.646854:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.646868:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.646883:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.646890:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760702.646895:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.646905:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.646998:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.647008:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d120 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.647105:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515588992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39975us (42582us total) trans 4294971951 rc 0/0 00000100:00100000:0.0:1789760702.647115:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4663 00000100:00100000:0.0:1789760702.647125:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681006307329 time 681072399125 state 0x2 ref 0 00000100:00100000:0.0:1789760702.652204:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.652300:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515589120 00000100:00100000:1.0:1789760702.652309:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515589120 00000100:00100000:1.0:1789760702.652363:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681006307329 time 681072399125 state 0x0 ref 1 00000100:00100000:1.0:1789760702.652379:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681072659541 time 681072399125 state 0x1 ref 1 00000100:00100000:1.0:1789760702.652407:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681072659541 time 681117696814 state 0x0 ref 1 00000100:00100000:1.0:1789760702.652419:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515589120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.652463:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18984960, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:1.0:1789760702.652471:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.652485:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760702.652490:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.652495:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.652508:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.719525:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971952 is committed 00002000:00010000:2.0:1789760702.719989:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18984960 -> 18989056 00010000:00010000:2.0:1789760702.720005:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.720017:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.720027:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.720035:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760702.720040:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.720053:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.720074:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.720082:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d127 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.720164:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515589120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67742us (67975us total) trans 4294971952 rc 0/0 00000100:00100000:2.0:1789760702.720174:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4664 00000100:00100000:2.0:1789760702.720184:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681072659541 time 681117696814 state 0x2 ref 0 00000100:00100000:2.0:1789760702.723861:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760702.762449:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971953 is committed 00000100:00100000:0.0:1789760702.772556:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:0.0:1789760702.782849:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760702.841464:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971954 is committed 00000100:00100000:2.0:1789760702.853151:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515589504 00000100:00100000:2.0:1789760702.853162:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515589504 00000100:00100000:2.0:1789760702.853232:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681189608139 time 681238574331 state 0x0 ref 1 00000100:00100000:2.0:1789760702.853248:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681238834747 time 681238574331 state 0x1 ref 1 00000100:00100000:2.0:1789760702.853289:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681238834747 time 681318577628 state 0x0 ref 1 00000100:00100000:2.0:1789760702.853302:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515589504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760702.853349:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 18997248, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9807 00010000:00010000:2.0:1789760702.853359:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.853375:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760702.853379:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.853385:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.853398:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760702.887587:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 18997248 -> 19001344 00002000:00010000:2.0:1789760702.887596:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760702 -> 1789760703 00002000:00010000:2.0:1789760702.887598:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760702 -> 1789760703 00010000:00010000:2.0:1789760702.887612:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.887625:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.887633:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.887639:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760702.887645:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.887659:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.887678:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.887685:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d13c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.887809:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515589504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34504us (34865us total) trans 4294971955 rc 0/0 00000100:00100000:2.0:1789760702.887820:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4667 00000100:00100000:2.0:1789760702.887831:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681238834747 time 681318577628 state 0x2 ref 0 00000001:00080000:0.0:1789760702.888615:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971955 is committed 00000100:00100000:0.0:1789760702.891751:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760702.921833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971956 is committed 00002000:00010000:0.0:1789760702.922011:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19001344 -> 19005440 00010000:00010000:0.0:1789760702.922028:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.922040:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.922049:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.922056:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760702.922061:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.922069:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.922090:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.922098:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d143 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.924889:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515589632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30112us (30490us total) trans 4294971956 rc 0/0 00000100:00100000:0.0:1789760702.925056:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4668 00000100:00100000:0.0:1789760702.925079:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681318838044 time 681357357311 state 0x2 ref 0 00000100:00100000:0.0:1789760702.926410:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760702.926479:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515589760 00000100:00100000:1.0:1789760702.926489:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515589760 00000100:00100000:1.0:1789760702.926539:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681318838044 time 681357357311 state 0x0 ref 1 00000100:00100000:1.0:1789760702.926553:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681357617727 time 681357357311 state 0x1 ref 1 00000100:00100000:1.0:1789760702.926584:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681357617727 time 681391872927 state 0x0 ref 1 00000100:00100000:1.0:1789760702.926596:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515589760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760702.926644:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19005440, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:1.0:1789760702.926654:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.926669:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760702.926672:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.926678:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.926691:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760702.971688:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19005440 -> 19009536 00010000:00010000:1.0:1789760702.971728:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.971741:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.971749:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.971756:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760702.971761:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.971769:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.971787:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.971794:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d14a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19005440->19009535] (req 19005440->19009535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.971885:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515589760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45285us (45474us total) trans 4294971957 rc 0/0 00000100:00100000:1.0:1789760702.971894:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4669 00000100:00100000:1.0:1789760702.971904:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681357617727 time 681391872927 state 0x2 ref 0 00000001:00080000:2.0:1789760702.975483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971957 is committed 00000100:00100000:2.0:1789760702.975519:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c25b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544404480:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760702.975590:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760702.975673:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404480 00000100:00100000:2.0:1789760702.975680:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404480 00000100:00100000:2.0:1789760702.975751:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1789760702.975760:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544404480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760702.975781:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3838836736 avail 3809476608 00000100:00100000:2.0:1789760702.975821:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544404480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760702.975830:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00080000:2.0:1789760702.975847:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760702.975902:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c25b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544404480:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760702.976822:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760702.976835:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544404608:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760702.976886:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760702.977059:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404608 00000100:00100000:1.0:1789760702.977065:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404608 00000100:00100000:1.0:1789760702.977110:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1789760702.977117:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544404608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760702.977142:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760702.977182:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544404608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.977201:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1789760702.978161:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00080000:0.0:1789760702.979291:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760702.979335:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544404608:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760703.023945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971958 is committed 00000100:00100000:0.0:1789760703.029096:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760703.073614:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19013632 -> 19017728 00010000:00010000:0.0:1789760703.073634:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.073647:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.073657:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.073664:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760703.073669:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.073678:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.073698:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.073727:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d158 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.073831:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515590016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44392us (44737us total) trans 4294971959 rc 0/0 00000100:00100000:0.0:1789760703.073841:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4671 00000100:00100000:0.0:1789760703.073850:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681443876043 time 681494712271 state 0x2 ref 0 00000001:00080000:1.0:1789760703.074028:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971959 is committed 00000100:00100000:0.0:1789760703.079835:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760703.129546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971960 is committed 00000100:00100000:1.0:1789760703.135782:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515590272 00000100:00100000:1.0:1789760703.135791:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515590272 00000100:00100000:1.0:1789760703.135845:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681494972687 time 681546106802 state 0x0 ref 1 00000100:00100000:1.0:1789760703.135861:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681546367218 time 681546106802 state 0x1 ref 1 00000100:00100000:1.0:1789760703.135888:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681546367218 time 681601177660 state 0x0 ref 1 00000100:00100000:1.0:1789760703.135900:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515590272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760703.136046:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19021824, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:1.0:1789760703.136056:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.136677:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760703.136683:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.136690:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.136736:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.174734:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971961 is committed 00002000:00010000:1.0:1789760703.174816:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19021824 -> 19025920 00010000:00010000:1.0:1789760703.174830:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.174850:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.174860:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.174867:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760703.174872:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.174881:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.174899:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.174908:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d166 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.175006:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515590272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39103us (39325us total) trans 4294971961 rc 0/0 00000100:00100000:1.0:1789760703.175016:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4673 00000100:00100000:1.0:1789760703.175025:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681546367218 time 681601177660 state 0x2 ref 0 00000100:00100000:1.0:1789760703.178792:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760703.178867:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515590400 00000100:00100000:0.0:1789760703.178876:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515590400 00000100:00100000:0.0:1789760703.178938:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681546367218 time 681601177660 state 0x0 ref 1 00000100:00100000:0.0:1789760703.178954:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681601438076 time 681601177660 state 0x1 ref 1 00000100:00100000:0.0:1789760703.178985:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681601438076 time 681644274620 state 0x0 ref 1 00000100:00100000:0.0:1789760703.178997:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515590400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760703.179044:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19025920, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:0.0:1789760703.179052:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.179072:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760703.179075:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.179081:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.179094:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.210042:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971962 is committed 00002000:00010000:0.0:1789760703.210196:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19025920 -> 19030016 00010000:00010000:0.0:1789760703.210212:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.210224:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.210232:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.210239:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760703.210244:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.210252:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.210280:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.210289:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d16d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.210377:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515590400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31376us (31588us total) trans 4294971962 rc 0/0 00000100:00100000:0.0:1789760703.210386:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4674 00000100:00100000:0.0:1789760703.210396:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681601438076 time 681644274620 state 0x2 ref 0 00000100:00100000:2.0:1789760703.214659:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760703.217452:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515590528 00000100:00100000:0.0:1789760703.217466:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515590528 00000100:00100000:1.0:1789760703.218097:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681601438076 time 681644274620 state 0x0 ref 1 00000100:00100000:1.0:1789760703.218114:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681644535036 time 681644274620 state 0x1 ref 1 00000100:00100000:1.0:1789760703.218138:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681644535036 time 681683427779 state 0x0 ref 1 00000100:00100000:1.0:1789760703.218150:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515590528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760703.218209:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19030016, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:1.0:1789760703.218218:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.218233:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760703.218238:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.218244:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.218509:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.256361:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19030016 -> 19034112 00010000:00010000:2.0:1789760703.256376:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.256389:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.256399:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.256406:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760703.256410:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.256418:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.256436:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.256444:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d174 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.256571:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515590528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38419us (41917us total) trans 4294971963 rc 0/0 00000100:00100000:2.0:1789760703.256581:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4675 00000100:00100000:2.0:1789760703.256591:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681644535036 time 681683427779 state 0x2 ref 0 00000100:00100000:1.0:1789760703.260471:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760703.303534:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760703.335163:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515590912 00000100:00100000:0.0:1789760703.335175:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515590912 00000100:00100000:0.0:1789760703.335241:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681726223618 time 681769165173 state 0x0 ref 1 00000100:00100000:0.0:1789760703.335256:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681769425589 time 681769165173 state 0x1 ref 1 00000100:00100000:0.0:1789760703.335295:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681769425589 time 681800583238 state 0x0 ref 1 00000100:00100000:0.0:1789760703.335308:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515590912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760703.335365:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19042304, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:0.0:1789760703.335375:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d189 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.335391:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760703.335395:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.335401:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.335416:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19042304->19046399] (req 19042304->19046399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.382433:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971966 is committed 00000100:00100000:0.0:1789760703.398124:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760703.398215:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515591040 00000100:00100000:1.0:1789760703.398225:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515591040 00000100:00100000:1.0:1789760703.398287:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681769425589 time 681800583238 state 0x0 ref 1 00000100:00100000:1.0:1789760703.398303:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681800843654 time 681800583238 state 0x1 ref 1 00000100:00100000:1.0:1789760703.398332:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681800843654 time 681863621495 state 0x0 ref 1 00000100:00100000:1.0:1789760703.398345:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515591040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760703.398392:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19046400, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:1.0:1789760703.398401:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.398416:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:1.0:1789760703.398420:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.398426:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.398440:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.441658:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19046400 -> 19050496 00010000:00010000:2.0:1789760703.441673:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.441685:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.441694:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.441701:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760703.441733:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.441742:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.441762:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.441774:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d190 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.441985:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515591040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43636us (43866us total) trans 4294971967 rc 0/0 00000100:00100000:2.0:1789760703.441996:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4679 00000100:00100000:2.0:1789760703.442007:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681800843654 time 681863621495 state 0x2 ref 0 00000100:00100000:0.0:1789760703.447088:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760703.447161:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515591168 00000100:00100000:2.0:1789760703.447172:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515591168 00000100:00100000:2.0:1789760703.447236:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681800843654 time 681863621495 state 0x0 ref 1 00000100:00100000:2.0:1789760703.447252:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681863881911 time 681863621495 state 0x1 ref 1 00000100:00100000:2.0:1789760703.447287:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681863881911 time 681912576803 state 0x0 ref 1 00000100:00100000:2.0:1789760703.447300:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515591168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760703.447350:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19050496, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:2.0:1789760703.447358:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d197 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.447375:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:2.0:1789760703.447379:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.447384:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.447403:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.487493:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515591296 00000100:00100000:2.0:1789760703.487509:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515591296 00000100:00100000:0.0:1789760703.488128:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681863881911 time 681912576803 state 0x0 ref 1 00000100:00100000:0.0:1789760703.488145:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681912837219 time 681912576803 state 0x1 ref 1 00000100:00100000:0.0:1789760703.488179:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681912837219 time 681953468972 state 0x0 ref 1 00000100:00100000:0.0:1789760703.488205:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760703.488257:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19054592, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:0.0:1789760703.488279:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.488298:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760703.488303:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.488308:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.488321:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760703.555622:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19054592 -> 19058688 00010000:00010000:1.0:1789760703.555639:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.555651:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.555660:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.555667:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760703.555671:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.555680:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.555699:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.555735:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d19e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.555820:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67615us (70169us total) trans 4294971969 rc 0/0 00000100:00100000:1.0:1789760703.555830:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4681 00000100:00100000:1.0:1789760703.555840:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681912837219 time 681953468972 state 0x2 ref 0 00000100:00100000:1.0:1789760703.560152:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760703.622000:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971970 is committed 00000100:00100000:2.0:1789760703.627494:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515591552 00000100:00100000:2.0:1789760703.627506:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515591552 00000100:00100000:2.0:1789760703.627574:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 681953729388 time 682026146744 state 0x0 ref 1 00000100:00100000:2.0:1789760703.627590:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682026407160 time 682026146744 state 0x1 ref 1 00000100:00100000:2.0:1789760703.627617:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682026407160 time 682092905905 state 0x0 ref 1 00000100:00100000:2.0:1789760703.627629:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760703.627678:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19062784, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:2.0:1789760703.627687:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.627702:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760703.627733:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.627741:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.627756:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.711876:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971971 is committed 00002000:00010000:2.0:1789760703.712055:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19062784 -> 19066880 00010000:00010000:2.0:1789760703.712070:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.712083:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.712091:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.712099:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760703.712104:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.712113:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.712133:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.712140:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.712457:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 84824us (85341us total) trans 4294971971 rc 0/0 00000100:00100000:2.0:1789760703.712472:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4683 00000100:00100000:2.0:1789760703.712482:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682026407160 time 682092905905 state 0x2 ref 0 00000100:00100000:2.0:1789760703.725846:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760703.790840:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971972 is committed 00002000:00010000:0.0:1789760703.791018:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19066880 -> 19070976 00010000:00010000:0.0:1789760703.791036:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.791050:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.791060:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.791068:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760703.791073:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.791082:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.791102:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.791109:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d1b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.791315:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64756us (65472us total) trans 4294971972 rc 0/0 00000100:00100000:0.0:1789760703.791329:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4684 00000100:00100000:0.0:1789760703.791338:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682093166321 time 682191832136 state 0x2 ref 0 00000100:00100000:2.0:1789760703.795167:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515591808 00000100:00100000:2.0:1789760703.795178:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515591808 00000100:00100000:2.0:1789760703.795231:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682093166321 time 682191832136 state 0x0 ref 1 00000100:00100000:2.0:1789760703.795248:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682192092552 time 682191832136 state 0x1 ref 1 00000100:00100000:2.0:1789760703.795380:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682192092552 time 682260667789 state 0x0 ref 1 00000100:00100000:2.0:1789760703.795394:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760703.795445:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19070976, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:2.0:1789760703.795454:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.795469:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760703.795474:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.795480:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.795507:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.823895:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971973 is committed 00002000:00010000:0.0:1789760703.824074:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19070976 -> 19075072 00010000:00010000:0.0:1789760703.824090:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.824102:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.824112:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.824119:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760703.824124:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.824131:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.824149:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.824156:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d1ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.824274:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515591808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28868us (29284us total) trans 4294971973 rc 0/0 00000100:00100000:0.0:1789760703.824285:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4685 00000100:00100000:0.0:1789760703.824294:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682192092552 time 682260667789 state 0x2 ref 0 00000100:00100000:1.0:1789760703.827396:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760703.828319:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515591936 00000100:00100000:2.0:1789760703.828330:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515591936 00000100:00100000:2.0:1789760703.828381:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682192092552 time 682260667789 state 0x0 ref 1 00000100:00100000:2.0:1789760703.828396:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682260928205 time 682260667789 state 0x1 ref 1 00000100:00100000:2.0:1789760703.828467:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682260928205 time 682293754959 state 0x0 ref 1 00000100:00100000:2.0:1789760703.828480:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515591936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760703.828530:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19075072, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9807 00010000:00010000:2.0:1789760703.828540:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.828555:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760703.828559:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.828565:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.828579:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.861618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971974 is committed 00002000:00010000:2.0:1789760703.861704:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19075072 -> 19079168 00002000:00010000:2.0:1789760703.861811:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760703 -> 1789760704 00002000:00010000:2.0:1789760703.861816:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760703 -> 1789760704 00010000:00010000:2.0:1789760703.861831:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.861843:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.861852:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.861858:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760703.861863:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.861871:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.861889:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.861896:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d1c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.862030:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515591936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33548us (34639us total) trans 4294971974 rc 0/0 00000100:00100000:2.0:1789760703.862040:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4686 00000100:00100000:2.0:1789760703.862049:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682260928205 time 682293754959 state 0x2 ref 0 00000100:00100000:2.0:1789760703.866276:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760703.866412:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592064 00000100:00100000:1.0:1789760703.866422:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592064 00000100:00100000:1.0:1789760703.866478:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682260928205 time 682293754959 state 0x0 ref 1 00000100:00100000:1.0:1789760703.866494:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682294015375 time 682293754959 state 0x1 ref 1 00000100:00100000:1.0:1789760703.866528:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682294015375 time 682331816821 state 0x0 ref 1 00000100:00100000:1.0:1789760703.866540:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515592064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760703.866592:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19079168, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760703.866602:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.866617:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760703.866621:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.866627:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.866642:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.908204:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971975 is committed 00002000:00010000:1.0:1789760703.909840:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19079168 -> 19083264 00010000:00010000:1.0:1789760703.909857:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.909870:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.909878:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.909885:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760703.909891:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.909900:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.909931:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.909938:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.910024:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515592064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43482us (43764us total) trans 4294971975 rc 0/0 00000100:00100000:1.0:1789760703.910035:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4687 00000100:00100000:1.0:1789760703.910045:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682294015375 time 682331816821 state 0x2 ref 0 00000100:00100000:0.0:1789760703.913165:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592192 00000100:00100000:0.0:1789760703.913177:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592192 00000100:00100000:0.0:1789760703.913400:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682294015375 time 682331816821 state 0x0 ref 1 00000100:00100000:0.0:1789760703.913415:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682332077237 time 682331816821 state 0x1 ref 1 00000100:00100000:0.0:1789760703.913463:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682332077237 time 682378753047 state 0x0 ref 1 00000100:00100000:0.0:1789760703.913473:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515592192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760703.913512:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19083264, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:0.0:1789760703.913518:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.913531:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760703.913534:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.913538:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.913548:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.942871:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971976 is committed 00002000:00010000:0.0:1789760703.943012:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19083264 -> 19087360 00010000:00010000:0.0:1789760703.943027:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.943039:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.943048:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.943056:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760703.943061:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.943070:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.943090:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.943098:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d1cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.943256:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515592192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29710us (30118us total) trans 4294971976 rc 0/0 00000100:00100000:0.0:1789760703.943281:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4688 00000100:00100000:0.0:1789760703.943292:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682332077237 time 682378753047 state 0x2 ref 0 00000100:00100000:0.0:1789760703.947501:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760703.948090:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682379013463 time 682413376524 state 0x0 ref 1 00000100:00100000:2.0:1789760703.948105:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515592320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760703.948152:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19087360, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760703.948161:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d1d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.948176:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760703.948181:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.948188:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d1d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.948215:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d1d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.972120:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971977 is committed 00000100:00100000:2.0:1789760703.979524:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760703.979660:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592448 00000100:00100000:0.0:1789760703.979669:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592448 00000100:00100000:0.0:1789760703.979756:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682379013463 time 682413376524 state 0x0 ref 1 00000100:00100000:0.0:1789760703.979772:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682413636940 time 682413376524 state 0x1 ref 1 00000100:00100000:0.0:1789760703.979797:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682413636940 time 682445085951 state 0x0 ref 1 00000100:00100000:0.0:1789760703.979810:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515592448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760703.979956:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19091456, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:0.0:1789760703.979966:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.979981:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760703.979985:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.979992:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.980006:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.028242:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19091456 -> 19095552 00010000:00010000:0.0:1789760704.028274:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.028288:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.028298:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.028305:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760704.028311:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.028320:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.028338:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.028346:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d1dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.028443:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515592448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48632us (48925us total) trans 4294971978 rc 0/0 00000100:00100000:0.0:1789760704.028453:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4690 00000100:00100000:0.0:1789760704.028462:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682413636940 time 682445085951 state 0x2 ref 0 00000001:00080000:1.0:1789760704.029805:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971978 is committed 00000100:00100000:0.0:1789760704.031768:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760704.031837:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592576 00000100:00100000:1.0:1789760704.031845:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592576 00000100:00100000:1.0:1789760704.031910:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682413636940 time 682445085951 state 0x0 ref 1 00000100:00100000:1.0:1789760704.031926:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682445346367 time 682445085951 state 0x1 ref 1 00000100:00100000:1.0:1789760704.031954:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682445346367 time 682497243719 state 0x0 ref 1 00000100:00100000:1.0:1789760704.031966:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515592576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.032013:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19095552, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.032022:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.032037:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760704.032041:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.032047:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.032062:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d1e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760704.032258:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760704.071087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971979 is committed 00000100:00100000:0.0:1789760704.075850:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760704.075946:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592704 00000100:00100000:1.0:1789760704.075956:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592704 00000100:00100000:1.0:1789760704.076007:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682445346367 time 682497243719 state 0x0 ref 1 00000100:00100000:1.0:1789760704.076022:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682497504135 time 682497243719 state 0x1 ref 1 00000100:00100000:1.0:1789760704.076053:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682497504135 time 682541342329 state 0x0 ref 1 00000100:00100000:1.0:1789760704.076064:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9093daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515592704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.076112:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19099648, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.076120:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.076142:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760704.076147:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.076153:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.076166:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.106331:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19099648 -> 19103744 00010000:00010000:2.0:1789760704.106348:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.106361:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.106374:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.106381:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760704.106385:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.106393:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.106410:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.106417:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d1eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.106509:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9093daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515592704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30442us (30663us total) trans 4294971980 rc 0/0 00000100:00100000:2.0:1789760704.106520:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4692 00000100:00100000:2.0:1789760704.106529:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682497504135 time 682541342329 state 0x2 ref 0 00000100:00100000:0.0:1789760704.111900:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760704.112097:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592832 00000100:00100000:1.0:1789760704.112107:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592832 00000100:00100000:1.0:1789760704.112157:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682497504135 time 682541342329 state 0x0 ref 1 00000100:00100000:1.0:1789760704.112171:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682541602745 time 682541342329 state 0x1 ref 1 00000100:00100000:1.0:1789760704.112343:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682541602745 time 682577630414 state 0x0 ref 1 00000100:00100000:1.0:1789760704.112356:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9093da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515592832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.112401:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19103744, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.112410:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.112425:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760704.112430:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.112437:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.112455:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.144901:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19103744 -> 19107840 00010000:00010000:2.0:1789760704.145057:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.145078:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.145087:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.145095:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760704.145100:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.145109:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.145129:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.145137:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d1f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.145685:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9093da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515592832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33326us (33789us total) trans 4294971981 rc 0/0 00000100:00100000:2.0:1789760704.145698:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4693 00000100:00100000:2.0:1789760704.145745:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682541602745 time 682577630414 state 0x2 ref 0 00000100:00100000:1.0:1789760704.150181:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515592960 00000100:00100000:1.0:1789760704.150201:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515592960 00000100:00100000:1.0:1789760704.150255:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682541602745 time 682577630414 state 0x0 ref 1 00000100:00100000:1.0:1789760704.150285:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682577890830 time 682577630414 state 0x1 ref 1 00000100:00100000:1.0:1789760704.150314:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682577890830 time 682615603895 state 0x0 ref 1 00000100:00100000:1.0:1789760704.150326:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515592960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.150371:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19107840, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.150379:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.150396:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760704.150401:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.150408:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.150422:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.184630:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19107840 -> 19111936 00010000:00010000:2.0:1789760704.184647:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.184661:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.184670:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.184677:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760704.184682:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.184691:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.184742:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.184751:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d1f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.184833:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515592960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34503us (34757us total) trans 4294971982 rc 0/0 00000100:00100000:2.0:1789760704.184842:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4694 00000100:00100000:2.0:1789760704.184853:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682577890830 time 682615603895 state 0x2 ref 0 00000100:00100000:0.0:1789760704.190105:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760704.233376:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760704.233524:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515593216 00000100:00100000:2.0:1789760704.233535:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515593216 00000100:00100000:2.0:1789760704.233588:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682615864311 time 682655762444 state 0x0 ref 1 00000100:00100000:2.0:1789760704.233609:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682656022860 time 682655762444 state 0x1 ref 1 00000100:00100000:2.0:1789760704.233639:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682656022860 time 682698928673 state 0x0 ref 1 00000100:00100000:2.0:1789760704.233650:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515593216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760704.233696:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19116032, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760704.233704:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d207 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.233748:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760704.233752:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.233758:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.233771:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.306497:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515593344 00000100:00100000:1.0:1789760704.306517:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515593344 00000100:00100000:1.0:1789760704.306572:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682656022860 time 682698928673 state 0x0 ref 1 00000100:00100000:1.0:1789760704.306588:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682699189089 time 682698928673 state 0x1 ref 1 00000100:00100000:1.0:1789760704.306618:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682699189089 time 682771908001 state 0x0 ref 1 00000100:00100000:1.0:1789760704.306630:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515593344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.306679:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19120128, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.306687:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.306700:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:1.0:1789760704.306747:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.306754:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.306768:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760704.363948:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19120128 -> 19124224 00010000:00010000:1.0:1789760704.363965:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.363977:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.363991:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.363999:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760704.364005:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.364014:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.364032:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.364040:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d20e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.364131:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515593344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57497us (57775us total) trans 4294971985 rc 0/0 00000100:00100000:1.0:1789760704.364142:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4697 00000100:00100000:1.0:1789760704.364152:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682699189089 time 682771908001 state 0x2 ref 0 00000100:00100000:0.0:1789760704.369992:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760704.370107:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515593472 00000100:00100000:2.0:1789760704.370116:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515593472 00000100:00100000:2.0:1789760704.370169:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682699189089 time 682771908001 state 0x0 ref 1 00000100:00100000:2.0:1789760704.370184:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682772168417 time 682771908001 state 0x1 ref 1 00000100:00100000:2.0:1789760704.370276:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682772168417 time 682835550900 state 0x0 ref 1 00000100:00100000:2.0:1789760704.370289:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515593472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760704.370344:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19124224, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760704.370353:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.370369:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760704.370373:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.370379:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.370392:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.421804:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19124224 -> 19128320 00010000:00010000:2.0:1789760704.421820:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.421832:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.421841:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.421848:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760704.421854:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.421864:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.421883:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.421891:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d215 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.422097:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515593472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51805us (52110us total) trans 4294971986 rc 0/0 00000100:00100000:2.0:1789760704.422108:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4698 00000100:00100000:2.0:1789760704.422118:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682772168417 time 682835550900 state 0x2 ref 0 00000100:00100000:0.0:1789760704.426249:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760704.426458:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515593600 00000100:00100000:2.0:1789760704.426469:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515593600 00000100:00100000:2.0:1789760704.426526:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682772168417 time 682835550900 state 0x0 ref 1 00000100:00100000:2.0:1789760704.426540:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682835811316 time 682835550900 state 0x1 ref 1 00000100:00100000:2.0:1789760704.426573:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682835811316 time 682891863023 state 0x0 ref 1 00000100:00100000:2.0:1789760704.426585:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cadc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515593600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760704.426636:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19128320, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760704.426646:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.426663:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760704.426667:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.426673:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.426687:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.497535:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19128320 -> 19132416 00010000:00010000:2.0:1789760704.497554:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.497566:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.497573:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.497580:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760704.497585:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.497594:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.497617:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.497624:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d21c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.497704:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cadc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515593600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71116us (71460us total) trans 4294971987 rc 0/0 00000100:00100000:2.0:1789760704.497748:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4699 00000100:00100000:2.0:1789760704.497759:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682835811316 time 682891863023 state 0x2 ref 0 00000001:00080000:2.0:1789760704.499561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971987 is committed 00000100:00100000:2.0:1789760704.503704:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515593728 00000100:00100000:2.0:1789760704.503754:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515593728 00000100:00100000:2.0:1789760704.503808:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682835811316 time 682891863023 state 0x0 ref 1 00000100:00100000:2.0:1789760704.503824:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682892123439 time 682891863023 state 0x1 ref 1 00000100:00100000:2.0:1789760704.503855:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682892123439 time 682969143889 state 0x0 ref 1 00000100:00100000:2.0:1789760704.503867:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515593728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760704.503911:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19132416, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760704.503920:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.503955:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=682s v=(0 0) 00010000:00010000:2.0:1789760704.503959:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.503965:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.503979:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.552387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971988 is committed 00002000:00010000:2.0:1789760704.552473:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19132416 -> 19136512 00010000:00010000:2.0:1789760704.552489:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.552503:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.552511:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.552518:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760704.552523:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.552530:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.552615:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.552691:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d223 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.552949:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515593728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49079us (49332us total) trans 4294971988 rc 0/0 00000100:00100000:2.0:1789760704.552961:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4700 00000100:00100000:2.0:1789760704.552970:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 682892123439 time 682969143889 state 0x2 ref 0 00000100:00100000:0.0:1789760704.558610:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760704.591745:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971989 is committed 00000100:00100000:0.0:1789760704.596236:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760704.638078:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971990 is committed 00000100:00100000:0.0:1789760704.641454:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760704.641551:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515594112 00000100:00100000:1.0:1789760704.641561:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515594112 00000100:00100000:1.0:1789760704.641611:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683025743076 time 683062485198 state 0x0 ref 1 00000100:00100000:1.0:1789760704.641626:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683062745614 time 683062485198 state 0x1 ref 1 00000100:00100000:1.0:1789760704.641742:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683062745614 time 683106992778 state 0x0 ref 1 00000100:00100000:1.0:1789760704.641758:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515594112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.641805:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19144704, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.641814:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.641834:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760704.641838:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.641845:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.641860:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760704.686277:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19144704 -> 19148800 00010000:00010000:1.0:1789760704.686293:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.686306:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.686321:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.686328:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760704.686333:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.686342:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.686362:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.686370:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d238 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.686556:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515594112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44796us (45106us total) trans 4294971991 rc 0/0 00000100:00100000:1.0:1789760704.686567:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4703 00000100:00100000:1.0:1789760704.686585:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683062745614 time 683106992778 state 0x2 ref 0 00000100:00100000:1.0:1789760704.691151:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760704.691290:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515594240 00000100:00100000:2.0:1789760704.691300:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515594240 00000100:00100000:2.0:1789760704.691353:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683062745614 time 683106992778 state 0x0 ref 1 00000100:00100000:2.0:1789760704.691369:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683107253194 time 683106992778 state 0x1 ref 1 00000100:00100000:2.0:1789760704.691398:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683107253194 time 683156687092 state 0x0 ref 1 00000100:00100000:2.0:1789760704.691409:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515594240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760704.691456:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19148800, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760704.691464:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d23f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.691479:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.691483:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.691489:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d23f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.691507:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d23f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19148800->19152895] (req 19148800->19152895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.746786:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760704.746990:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515594368 00000100:00100000:2.0:1789760704.746999:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515594368 00000100:00100000:2.0:1789760704.747055:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683107253194 time 683156687092 state 0x0 ref 1 00000100:00100000:2.0:1789760704.747076:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683156947508 time 683156687092 state 0x1 ref 1 00000100:00100000:2.0:1789760704.747100:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683156947508 time 683212389496 state 0x0 ref 1 00000100:00100000:2.0:1789760704.747111:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515594368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760704.747151:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19152896, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:2.0:1789760704.747160:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d246 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.747182:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.747187:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.747193:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.747206:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.790750:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971993 is committed 00000100:00100000:2.0:1789760704.795195:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760704.795276:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515594496 00000100:00100000:1.0:1789760704.795286:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515594496 00000100:00100000:1.0:1789760704.795339:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683156947508 time 683212389496 state 0x0 ref 1 00000100:00100000:1.0:1789760704.795355:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683212649912 time 683212389496 state 0x1 ref 1 00000100:00100000:1.0:1789760704.795447:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683212649912 time 683260734672 state 0x0 ref 1 00000100:00100000:1.0:1789760704.795462:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515594496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.795512:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19156992, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9807 00010000:00010000:1.0:1789760704.795520:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.795534:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760704.795539:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.795546:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.795558:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.829844:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760704.843064:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19156992 -> 19161088 00010000:00010000:1.0:1789760704.843080:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.843093:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.843102:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.843110:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760704.843115:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.843124:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.843144:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.843152:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d24d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.843322:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515594496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47857us (48131us total) trans 4294971994 rc 0/0 00000100:00100000:1.0:1789760704.843333:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4706 00000100:00100000:1.0:1789760704.843344:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683212649912 time 683260734672 state 0x2 ref 0 00000100:00100000:0.0:1789760704.847206:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760704.889781:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760704.941741:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971996 is committed 00000100:00100000:2.0:1789760704.945881:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760704.946032:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515594880 00000100:00100000:1.0:1789760704.946041:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515594880 00000100:00100000:1.0:1789760704.946094:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683313035711 time 683355415706 state 0x0 ref 1 00000100:00100000:1.0:1789760704.946111:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683355676122 time 683355415706 state 0x1 ref 1 00000100:00100000:1.0:1789760704.946143:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683355676122 time 683411432296 state 0x0 ref 1 00000100:00100000:1.0:1789760704.946164:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515594880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760704.946405:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19169280, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:1.0:1789760704.946415:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d262 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.946430:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760704.946435:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.946441:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.946455:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.973964:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971997 is committed 00000100:00100000:0.0:1789760704.977307:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760704.997207:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19173376 -> 19177472 00010000:00010000:1.0:1789760704.997222:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.997236:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.997245:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.997252:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760704.997269:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.997279:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.997298:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.997307:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d269 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.997393:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515595008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 19804us (20085us total) trans 4294971998 rc 0/0 00000100:00100000:1.0:1789760704.997404:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4710 00000100:00100000:1.0:1789760704.997413:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683411692712 time 683442855849 state 0x2 ref 0 00000100:00100000:2.0:1789760705.001889:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760705.038447:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19177472 -> 19181568 00010000:00010000:0.0:1789760705.038469:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.038482:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.038491:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.038503:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760705.038508:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.038518:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.038537:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.038544:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d270 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.038637:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515595136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36543us (36752us total) trans 4294971999 rc 0/0 00000100:00100000:0.0:1789760705.038648:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4711 00000100:00100000:0.0:1789760705.038658:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683443116265 time 683467370323 state 0x2 ref 0 00000001:00080000:1.0:1789760705.038734:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971999 is committed 00000100:00100000:2.0:1789760705.046218:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760705.046292:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515595264 00000100:00100000:1.0:1789760705.046302:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515595264 00000100:00100000:1.0:1789760705.046364:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683443116265 time 683467370323 state 0x0 ref 1 00000100:00100000:1.0:1789760705.046379:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683467630739 time 683467370323 state 0x1 ref 1 00000100:00100000:1.0:1789760705.046410:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683467630739 time 683511699313 state 0x0 ref 1 00000100:00100000:1.0:1789760705.046423:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515595264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760705.046471:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19181568, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:1.0:1789760705.046482:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.046498:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760705.046502:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.046509:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.046522:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.071153:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972000 is committed 00002000:00010000:1.0:1789760705.071169:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19181568 -> 19185664 00010000:00010000:1.0:1789760705.071184:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.071198:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.071207:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.071214:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760705.071219:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.071238:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.071275:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.071283:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d277 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.071371:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515595264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24946us (25159us total) trans 4294972000 rc 0/0 00000100:00100000:1.0:1789760705.071381:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4712 00000100:00100000:1.0:1789760705.071391:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683467630739 time 683511699313 state 0x2 ref 0 00000100:00100000:2.0:1789760705.076243:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515595392 00000100:00100000:2.0:1789760705.076262:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515595392 00000100:00100000:2.0:1789760705.076315:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683467630739 time 683511699313 state 0x0 ref 1 00000100:00100000:2.0:1789760705.076342:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683511959729 time 683511699313 state 0x1 ref 1 00000100:00100000:2.0:1789760705.076372:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683511959729 time 683541661329 state 0x0 ref 1 00000100:00100000:2.0:1789760705.076385:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515595392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760705.076431:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19185664, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:2.0:1789760705.076441:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.076457:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760705.076461:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.076468:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.076484:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760705.108247:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19185664 -> 19189760 00010000:00010000:0.0:1789760705.108276:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.108289:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.108304:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.108312:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760705.108318:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.108327:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.108347:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.108354:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d27e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19185664->19189759] (req 19185664->19189759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.108588:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515595392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32200us (32452us total) trans 4294972001 rc 0/0 00000100:00100000:0.0:1789760705.108599:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4713 00000100:00100000:0.0:1789760705.108608:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683511959729 time 683541661329 state 0x2 ref 0 00000100:00100000:2.0:1789760705.115026:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760705.115267:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515595520 00000100:00100000:0.0:1789760705.115277:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515595520 00000100:00100000:0.0:1789760705.115394:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683511959729 time 683541661329 state 0x0 ref 1 00000100:00100000:0.0:1789760705.115411:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683541921745 time 683541661329 state 0x1 ref 1 00000100:00100000:0.0:1789760705.115444:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683541921745 time 683580733415 state 0x0 ref 1 00000100:00100000:0.0:1789760705.115457:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515595520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760705.115904:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19189760, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:0.0:1789760705.115915:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d285 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.115931:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760705.115936:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.115942:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.115957:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.170869:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760705.226665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972003 is committed 00000100:00100000:2.0:1789760705.229444:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760705.229490:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515595776 00000100:00100000:1.0:1789760705.229499:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515595776 00000100:00100000:1.0:1789760705.229537:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683580993831 time 683636349083 state 0x0 ref 1 00000100:00100000:1.0:1789760705.229549:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683636609499 time 683636349083 state 0x1 ref 1 00000100:00100000:1.0:1789760705.229583:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683636609499 time 683694872289 state 0x0 ref 1 00000100:00100000:1.0:1789760705.229596:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515595776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760705.229640:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19197952, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:1.0:1789760705.229647:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.229665:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760705.229671:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.229678:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.229691:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760705.254171:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19197952 -> 19202048 00010000:00010000:2.0:1789760705.254195:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.254209:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.254218:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.254226:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760705.254231:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.254251:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.254430:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.254437:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d293 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19197952->19202047] (req 19197952->19202047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.254780:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515595776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25180us (25334us total) trans 4294972004 rc 0/0 00000100:00100000:2.0:1789760705.254791:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4716 00000100:00100000:2.0:1789760705.254801:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683636609499 time 683694872289 state 0x2 ref 0 00000100:00100000:2.0:1789760705.262894:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760705.320187:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972005 is committed 00002000:00010000:2.0:1789760705.320270:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19202048 -> 19206144 00010000:00010000:2.0:1789760705.320288:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.320301:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.320310:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.320317:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760705.320322:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.320329:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.320347:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.320355:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d29a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.320660:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515595904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57558us (57770us total) trans 4294972005 rc 0/0 00000100:00100000:2.0:1789760705.320672:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4717 00000100:00100000:2.0:1789760705.320682:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683695132705 time 683728375781 state 0x2 ref 0 00000100:00100000:0.0:1789760705.329044:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760705.329138:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515596032 00000100:00100000:1.0:1789760705.329147:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515596032 00000100:00100000:1.0:1789760705.329199:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683695132705 time 683728375781 state 0x0 ref 1 00000100:00100000:1.0:1789760705.329211:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683728636197 time 683728375781 state 0x1 ref 1 00000100:00100000:1.0:1789760705.329237:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683728636197 time 683794527079 state 0x0 ref 1 00000100:00100000:1.0:1789760705.329245:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515596032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760705.329303:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19206144, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:1.0:1789760705.329312:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.329328:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:1.0:1789760705.329333:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.329340:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.329354:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.361459:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972006 is committed 00002000:00010000:2.0:1789760705.361606:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19206144 -> 19210240 00010000:00010000:2.0:1789760705.361629:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.361640:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.361647:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.361655:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760705.361660:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.361669:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.361703:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.361780:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d2a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.361910:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515596032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32660us (32871us total) trans 4294972006 rc 0/0 00000100:00100000:2.0:1789760705.361934:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4718 00000100:00100000:2.0:1789760705.361942:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683728636197 time 683794527079 state 0x2 ref 0 00000100:00100000:0.0:1789760705.366249:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515596160 00000100:00100000:0.0:1789760705.366273:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515596160 00000100:00100000:0.0:1789760705.366331:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683728636197 time 683794527079 state 0x0 ref 1 00000100:00100000:0.0:1789760705.366346:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683794787495 time 683794527079 state 0x1 ref 1 00000100:00100000:0.0:1789760705.366374:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683794787495 time 683831663711 state 0x0 ref 1 00000100:00100000:0.0:1789760705.366387:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515596160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760705.366433:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19210240, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:0.0:1789760705.366443:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.366459:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760705.366463:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.366470:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.366484:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.388413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972007 is committed 00002000:00010000:2.0:1789760705.388510:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19210240 -> 19214336 00010000:00010000:2.0:1789760705.388525:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.388539:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.388548:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.388554:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760705.388559:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.388568:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.388588:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.388601:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d2a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.388697:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515596160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22308us (22544us total) trans 4294972007 rc 0/0 00000100:00100000:2.0:1789760705.388740:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4719 00000100:00100000:2.0:1789760705.388749:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683794787495 time 683831663711 state 0x2 ref 0 00000100:00100000:2.0:1789760705.392491:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760705.392564:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515596288 00000100:00100000:0.0:1789760705.392574:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515596288 00000100:00100000:0.0:1789760705.392625:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683794787495 time 683831663711 state 0x0 ref 1 00000100:00100000:0.0:1789760705.392641:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683831924127 time 683831663711 state 0x1 ref 1 00000100:00100000:0.0:1789760705.392694:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683831924127 time 683857982933 state 0x0 ref 1 00000100:00100000:0.0:1789760705.392704:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515596288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760705.392765:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19214336, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:0.0:1789760705.392773:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.392789:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760705.392794:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.392801:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.392815:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760705.432668:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19214336 -> 19218432 00010000:00010000:1.0:1789760705.432689:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.432702:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.432739:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.432747:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760705.432752:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.432763:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.432788:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.432796:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d2af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.433075:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515596288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40364us (40587us total) trans 4294972008 rc 0/0 00000100:00100000:1.0:1789760705.433087:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4720 00000100:00100000:1.0:1789760705.433097:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683831924127 time 683857982933 state 0x2 ref 0 00000001:00080000:1.0:1789760705.433137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972008 is committed 00000100:00100000:1.0:1789760705.437611:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760705.496247:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972009 is committed 00002000:00010000:1.0:1789760705.496408:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19218432 -> 19222528 00010000:00010000:1.0:1789760705.496424:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.496437:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.496447:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.496455:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760705.496461:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.496471:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.496491:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.496501:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d2b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.496595:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515596416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58324us (58989us total) trans 4294972009 rc 0/0 00000100:00100000:1.0:1789760705.496606:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4721 00000100:00100000:1.0:1789760705.496617:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683858243349 time 683903332926 state 0x2 ref 0 00000100:00100000:2.0:1789760705.500750:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760705.536486:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972010 is committed 00000100:00100000:2.0:1789760705.546999:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:2.0:1789760705.591295:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19226624 -> 19230720 00010000:00010000:2.0:1789760705.591311:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.591324:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.591334:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.591347:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760705.591352:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.591361:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.591380:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.591388:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d2c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.591473:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515596672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43683us (44478us total) trans 4294972011 rc 0/0 00000100:00100000:2.0:1789760705.591483:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4723 00000100:00100000:2.0:1789760705.591493:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683966532314 time 684013062254 state 0x2 ref 0 00000100:00100000:2.0:1789760705.595695:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760705.596275:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515596800 00000100:00100000:1.0:1789760705.596285:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515596800 00000100:00100000:1.0:1789760705.596750:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 683966532314 time 684013062254 state 0x0 ref 1 00000100:00100000:1.0:1789760705.596770:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684013322670 time 684013062254 state 0x1 ref 1 00000100:00100000:0.0:1789760705.596956:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684013322670 time 684062242555 state 0x0 ref 1 00000100:00100000:0.0:1789760705.596971:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515596800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760705.597021:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19230720, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:0.0:1789760705.597031:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.597046:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760705.597051:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.597057:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.597071:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760705.649619:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19230720 -> 19234816 00010000:00010000:0.0:1789760705.649638:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649652:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649662:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649670:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760705.649674:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649684:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649702:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.649737:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19230720->19234815] (req 19230720->19234815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.649813:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515596800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52842us (54122us total) trans 4294972012 rc 0/0 00000100:00100000:0.0:1789760705.649823:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4724 00000100:00100000:0.0:1789760705.649832:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684013322670 time 684062242555 state 0x2 ref 0 00000100:00100000:2.0:1789760705.653317:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760705.654874:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684013322670 time 684062242555 state 0x0 ref 1 00000100:00100000:2.0:1789760705.654890:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684062502971 time 684062242555 state 0x1 ref 1 00000100:00100000:2.0:1789760705.654925:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684062502971 time 684120214411 state 0x0 ref 1 00000100:00100000:2.0:1789760705.654938:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515596928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760705.654983:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19234816, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:2.0:1789760705.654994:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.655009:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760705.655014:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.655020:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.655035:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760705.683881:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972013 is committed 00002000:00010000:2.0:1789760705.683945:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19234816 -> 19238912 00010000:00010000:2.0:1789760705.683960:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.683974:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.683983:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.683991:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760705.683996:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.684007:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.684028:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.684035:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d2d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.684113:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515596928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29173us (30801us total) trans 4294972013 rc 0/0 00000100:00100000:2.0:1789760705.684124:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4725 00000100:00100000:2.0:1789760705.684135:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684062502971 time 684120214411 state 0x2 ref 0 00000100:00100000:2.0:1789760705.691332:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760705.691440:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597056 00000100:00100000:0.0:1789760705.691450:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597056 00000100:00100000:0.0:1789760705.691500:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684062502971 time 684120214411 state 0x0 ref 1 00000100:00100000:0.0:1789760705.691514:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684120474827 time 684120214411 state 0x1 ref 1 00000100:00100000:0.0:1789760705.691574:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684120474827 time 684156862860 state 0x0 ref 1 00000100:00100000:0.0:1789760705.691586:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515597056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760705.691630:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19238912, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:0.0:1789760705.691638:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.691653:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760705.691657:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.691664:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.691677:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19238912->19243007] (req 19238912->19243007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.748655:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972014 is committed 00000100:00100000:2.0:1789760705.753981:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760705.754101:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597184 00000100:00100000:1.0:1789760705.754111:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597184 00000100:00100000:1.0:1789760705.754163:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684120474827 time 684156862860 state 0x0 ref 1 00000100:00100000:1.0:1789760705.754359:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684157123276 time 684156862860 state 0x1 ref 1 00000100:00100000:1.0:1789760705.754439:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684157123276 time 684219727788 state 0x0 ref 1 00000100:00100000:1.0:1789760705.754453:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515597184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760705.754499:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19243008, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:1.0:1789760705.754508:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.754524:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760705.754528:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.754535:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.754549:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760705.787521:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19243008 -> 19247104 00010000:00010000:1.0:1789760705.787535:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.787548:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.787558:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.787565:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760705.787570:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.787579:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.787600:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.787608:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760705.787703:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515597184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33240us (33719us total) trans 4294972015 rc 0/0 00000100:00100000:1.0:1789760705.787747:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4727 00000100:00100000:1.0:1789760705.787756:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684157123276 time 684219727788 state 0x2 ref 0 00000100:00100000:2.0:1789760705.796082:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760705.833328:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597440 00000100:00100000:1.0:1789760705.833338:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597440 00000100:00100000:1.0:1789760705.833389:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684219988204 time 684261580341 state 0x0 ref 1 00000100:00100000:1.0:1789760705.833403:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684261840757 time 684261580341 state 0x1 ref 1 00000100:00100000:1.0:1789760705.833433:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684261840757 time 684298722646 state 0x0 ref 1 00000100:00100000:1.0:1789760705.833444:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515597440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760705.833495:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19251200, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9807 00010000:00010000:1.0:1789760705.833504:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.833520:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760705.833524:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.833530:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.833550:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d2ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19251200->19255295] (req 19251200->19255295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.854055:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760705.909671:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972017 is committed 00000100:00100000:2.0:1789760705.924809:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760705.924900:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597568 00000100:00100000:0.0:1789760705.924911:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597568 00000100:00100000:0.0:1789760705.924963:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684261840757 time 684298722646 state 0x0 ref 1 00000100:00100000:0.0:1789760705.924979:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684298983062 time 684298722646 state 0x1 ref 1 00000100:00100000:0.0:1789760705.925010:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684298983062 time 684390299355 state 0x0 ref 1 00000100:00100000:0.0:1789760705.925022:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515597568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760705.925071:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19255296, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:0.0:1789760705.925080:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d2f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.925095:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760705.925100:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.925106:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d2f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.925121:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d2f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.000443:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972018 is committed 00000100:00100000:0.0:1789760706.006820:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760706.006908:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597696 00000100:00100000:2.0:1789760706.006919:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597696 00000100:00080000:0.0:1789760706.007060:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760706.007107:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684298983062 time 684390299355 state 0x0 ref 1 00000100:00100000:2.0:1789760706.007123:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684390559771 time 684390299355 state 0x1 ref 1 00000100:00100000:2.0:1789760706.007155:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684390559771 time 684472443855 state 0x0 ref 1 00000100:00100000:2.0:1789760706.007168:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515597696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760706.007224:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19259392, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:2.0:1789760706.007235:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d2fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.007260:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760706.007265:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.007271:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d2fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.007287:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d2fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.046028:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92f436680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544404864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.046091:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760706.046145:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544405376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.046170:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760706.046207:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760706.046213:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c62a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544404992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.046232:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760706.046274:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404864 00000100:00100000:0.0:1789760706.046281:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404864 00000100:00100000:0.0:1789760706.046322:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404992 00000100:00100000:0.0:1789760706.046325:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404992 00000100:00100000:0.0:1789760706.046354:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1789760706.046362:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544404864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.046414:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544404864:12345-0@lo:400:kworker.0 Request processed in 51us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760706.046421:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1789760706.046436:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:0.0:1789760706.046444:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544404992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.046468:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c3bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544404736:0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.046473:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544404992:12345-0@lo:400:kworker.0 Request processed in 29us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760706.046480:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00080000:0.0:1789760706.046504:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760706.046512:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760706.046538:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c62a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544404992:0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.046546:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c62300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544405248:0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.046552:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92f436680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544404864:0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.046573:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760706.046598:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760706.046605:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c60e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544405120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.046626:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760706.046650:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544404736 00000100:00100000:1.0:1789760706.046653:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544404736 00000100:00100000:1.0:1789760706.046677:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:1.0:1789760706.046683:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9092dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544404736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.046736:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9092dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544404736:12345-0@lo:400:kworker.0 Request processed in 53us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760706.046743:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:1.0:1789760706.046767:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544405376 00000100:00100000:1.0:1789760706.046771:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544405376 00000100:00100000:1.0:1789760706.046796:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544405120 00000100:00100000:1.0:1789760706.046800:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544405120 00000100:00100000:1.0:1789760706.046819:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1789760706.046825:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544405376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.046862:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544405376:12345-0@lo:400:kworker.0 Request processed in 37us (690us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760706.046868:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:0.0:1789760706.046979:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544405248 00000100:00100000:0.0:1789760706.046983:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544405248 00000100:00100000:0.0:1789760706.047009:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1789760706.047015:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544405248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760706.047050:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544405248:12345-0@lo:400:kworker.0 Request processed in 35us (479us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760706.047056:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00080000:0.0:1789760706.047074:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760706.047077:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760706.047078:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760706.047096:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544405376:0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.047805:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c60e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544405120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.047816:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c3bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544404736:0@lo:400:kworker.0 00000100:00100000:1.0:1789760706.047826:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c62300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544405248:0@lo:400:kworker.0 00000001:00080000:1.0:1789760706.080673:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972019 is committed 00000100:00100000:0.0:1789760706.084601:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597824 00000100:00100000:0.0:1789760706.084610:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597824 00000100:00100000:0.0:1789760706.084681:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684390559771 time 684472443855 state 0x0 ref 1 00000100:00100000:0.0:1789760706.084701:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684472704271 time 684472443855 state 0x1 ref 1 00000100:00100000:1.0:1789760706.085013:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684472704271 time 684550300094 state 0x0 ref 1 00000100:00100000:1.0:1789760706.085030:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515597824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.085081:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19263488, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.085090:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d303 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.085105:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760706.085109:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.085116:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.085130:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19263488->19267583] (req 19263488->19267583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.115630:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972020 is committed 00000100:00100000:2.0:1789760706.120216:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.120318:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515597952 00000100:00100000:1.0:1789760706.120329:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515597952 00000100:00100000:1.0:1789760706.120383:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684472704271 time 684550300094 state 0x0 ref 1 00000100:00100000:1.0:1789760706.120397:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684550560510 time 684550300094 state 0x1 ref 1 00000100:00100000:1.0:1789760706.120458:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684550560510 time 684585746440 state 0x0 ref 1 00000100:00100000:1.0:1789760706.120471:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515597952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.120517:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19267584, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.120527:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.120542:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760706.120547:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.120554:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.120568:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760706.151464:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19267584 -> 19271680 00010000:00010000:2.0:1789760706.151478:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.151540:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.151550:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.151557:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760706.151562:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.151571:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.151590:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.151598:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d30a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.151783:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515597952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31310us (31573us total) trans 4294972021 rc 0/0 00000100:00100000:2.0:1789760706.151793:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4733 00000100:00100000:2.0:1789760706.151803:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684550560510 time 684585746440 state 0x2 ref 0 00000100:00100000:0.0:1789760706.155400:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760706.196292:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19271680 -> 19275776 00010000:00010000:0.0:1789760706.196307:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.196320:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.196330:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.196337:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760706.196341:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.196350:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.196370:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.196377:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d311 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.196475:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515598080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40784us (41078us total) trans 4294972022 rc 0/0 00000100:00100000:0.0:1789760706.197827:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4734 00000100:00100000:0.0:1789760706.197842:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684586006856 time 684620962493 state 0x2 ref 0 00000100:00100000:1.0:1789760706.202431:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515598208 00000100:00100000:1.0:1789760706.202440:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515598208 00000100:00100000:1.0:1789760706.202492:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684586006856 time 684620962493 state 0x0 ref 1 00000100:00100000:1.0:1789760706.202506:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684621222909 time 684620962493 state 0x1 ref 1 00000100:00100000:1.0:1789760706.202535:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684621222909 time 684667825133 state 0x0 ref 1 00000100:00100000:1.0:1789760706.202551:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515598208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.202596:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19275776, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.202605:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d318 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.202620:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760706.202624:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.202630:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.202644:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.246658:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972023 is committed 00000100:00100000:2.0:1789760706.251764:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760706.252267:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19279872, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:0.0:1789760706.252338:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.252356:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:0.0:1789760706.252360:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.252370:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.252385:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760706.284843:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19279872 -> 19283968 00010000:00010000:1.0:1789760706.284862:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.284876:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.284897:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.284906:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.284911:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.284921:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.284941:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.284950:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d31f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.285046:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515598336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33054us (33332us total) trans 4294972024 rc 0/0 00000100:00100000:1.0:1789760706.285056:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4736 00000100:00100000:1.0:1789760706.285066:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684668085549 time 684717264408 state 0x2 ref 0 00000100:00100000:0.0:1789760706.288816:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760706.289507:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515598464 00000100:00100000:2.0:1789760706.289519:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515598464 00000100:00100000:2.0:1789760706.289575:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684668085549 time 684717264408 state 0x0 ref 1 00000100:00100000:2.0:1789760706.289590:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684717524824 time 684717264408 state 0x1 ref 1 00000100:00100000:2.0:1789760706.289661:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684717524824 time 684754949786 state 0x0 ref 1 00000100:00100000:2.0:1789760706.289674:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515598464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760706.289758:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19283968, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:2.0:1789760706.289767:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.289789:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760706.289793:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.289799:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.289823:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760706.311456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972025 is committed 00002000:00010000:0.0:1789760706.311533:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19283968 -> 19288064 00010000:00010000:0.0:1789760706.311548:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.311561:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.311575:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.311584:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760706.311589:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.311659:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.311678:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.311686:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d326 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.311830:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515598464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22152us (23027us total) trans 4294972025 rc 0/0 00000100:00100000:0.0:1789760706.311839:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4737 00000100:00100000:0.0:1789760706.311857:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684717524824 time 684754949786 state 0x2 ref 0 00000100:00100000:2.0:1789760706.315196:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760706.347232:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19288064 -> 19292160 00010000:00010000:1.0:1789760706.347265:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.347279:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.347289:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.347300:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.347304:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.347314:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.347333:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.347341:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d32d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.347461:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515598592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32042us (32267us total) trans 4294972026 rc 0/0 00000100:00100000:1.0:1789760706.347472:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4738 00000100:00100000:1.0:1789760706.347482:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684755210202 time 684780692286 state 0x2 ref 0 00000001:00080000:1.0:1789760706.347520:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972026 is committed 00000100:00100000:2.0:1789760706.350800:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.351015:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515598720 00000100:00100000:1.0:1789760706.351025:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515598720 00000100:00100000:1.0:1789760706.351077:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684755210202 time 684780692286 state 0x0 ref 1 00000100:00100000:1.0:1789760706.351092:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684780952702 time 684780692286 state 0x1 ref 1 00000100:00100000:1.0:1789760706.351180:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684780952702 time 684816468264 state 0x0 ref 1 00000100:00100000:1.0:1789760706.351192:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515598720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.351239:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19292160, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.351256:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.351272:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:1.0:1789760706.351276:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.351282:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.351297:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760706.382254:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19292160 -> 19296256 00010000:00010000:1.0:1789760706.382276:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.382290:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.382299:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.382311:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.382316:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.382327:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.382346:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.382354:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d334 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.382440:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515598720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31245us (31644us total) trans 4294972027 rc 0/0 00000100:00100000:1.0:1789760706.382452:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4739 00000100:00100000:1.0:1789760706.382461:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684780952702 time 684816468264 state 0x2 ref 0 00000100:00100000:0.0:1789760706.386418:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760706.425435:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972028 is committed 00000100:00100000:2.0:1789760706.429230:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760706.462670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972029 is committed 00002000:00010000:1.0:1789760706.462818:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19300352 -> 19304448 00010000:00010000:1.0:1789760706.462833:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.462856:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.462866:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.462878:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.462883:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.462902:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.462921:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.462930:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d342 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.463021:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515598976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33532us (33795us total) trans 4294972029 rc 0/0 00000100:00100000:1.0:1789760706.463033:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4741 00000100:00100000:1.0:1789760706.463042:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684852576325 time 684894761415 state 0x2 ref 0 00000100:00100000:2.0:1789760706.467456:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760706.467556:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515599104 00000100:00100000:0.0:1789760706.467566:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515599104 00000100:00100000:0.0:1789760706.467685:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684852576325 time 684894761415 state 0x0 ref 1 00000100:00100000:0.0:1789760706.467703:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684895021831 time 684894761415 state 0x1 ref 1 00000100:00100000:2.0:1789760706.468086:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684895021831 time 684933371302 state 0x0 ref 1 00000100:00100000:2.0:1789760706.468101:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806193800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515599104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760706.468146:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19304448, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:2.0:1789760706.468154:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.468179:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760706.468184:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.468190:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.468210:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.481948:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760706.490415:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760706.490496:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.495266:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515599360 00000100:00100000:1.0:1789760706.495280:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515599360 00000100:00100000:1.0:1789760706.495359:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515599488 00000100:00100000:1.0:1789760706.495363:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515599488 00000100:00100000:1.0:1789760706.495396:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 421 00000100:00100000:1.0:1789760706.495407:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515599360:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760706.495487:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515599360:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 82us (5082us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760706.495495:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 421 00000100:00100000:1.0:1789760706.495510:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 422 00000100:00100000:1.0:1789760706.495517:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515599488:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760706.495541:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515599488:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 25us (5045us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760706.495548:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 422 00000100:00080000:1.0:1789760706.495565:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760706.514291:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972030 is committed 00002000:00010000:1.0:1789760706.514449:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19304448 -> 19308544 00010000:00010000:1.0:1789760706.514463:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.514472:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.514478:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.514482:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.514485:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.514492:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.514506:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.514512:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d349 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.514584:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806193800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515599104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46483us (47134us total) trans 4294972030 rc 0/0 00000100:00100000:1.0:1789760706.514594:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4742 00000100:00100000:1.0:1789760706.514603:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684895021831 time 684933371302 state 0x2 ref 0 00000100:00100000:0.0:1789760706.517676:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760706.517986:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515599616 00000100:00100000:2.0:1789760706.517996:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515599616 00002000:00010000:0.0:1789760706.562230:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19308544 -> 19312640 00010000:00010000:0.0:1789760706.562258:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.562271:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.562281:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.562289:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760706.562294:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.562303:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.562324:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.562342:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d350 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.562436:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515599616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44210us (44764us total) trans 4294972031 rc 0/0 00000100:00100000:0.0:1789760706.562446:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4743 00000100:00100000:0.0:1789760706.562456:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684933631718 time 684983498911 state 0x2 ref 0 00000001:00080000:1.0:1789760706.564114:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972031 is committed 00000100:00100000:2.0:1789760706.568794:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.568915:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515599744 00000100:00100000:1.0:1789760706.568925:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515599744 00000100:00100000:1.0:1789760706.568976:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684933631718 time 684983498911 state 0x0 ref 1 00000100:00100000:1.0:1789760706.568991:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684983759327 time 684983498911 state 0x1 ref 1 00000100:00100000:1.0:1789760706.569021:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684983759327 time 685034310560 state 0x0 ref 1 00000100:00100000:1.0:1789760706.569033:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515599744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.569077:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19312640, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.569084:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.569100:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760706.569104:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.569110:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.569123:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760706.592572:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19312640 -> 19316736 00000001:00080000:2.0:1789760706.592576:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972032 is committed 00010000:00010000:1.0:1789760706.592590:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.592602:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.592612:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.592620:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.592626:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.592636:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.592656:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.592665:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d357 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.592906:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515599744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23868us (24115us total) trans 4294972032 rc 0/0 00000100:00100000:1.0:1789760706.592917:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4744 00000100:00100000:1.0:1789760706.592926:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684983759327 time 685034310560 state 0x2 ref 0 00000100:00100000:2.0:1789760706.595579:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.595665:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515599872 00000100:00100000:1.0:1789760706.595677:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515599872 00000100:00100000:1.0:1789760706.595807:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 684983759327 time 685034310560 state 0x0 ref 1 00000100:00100000:1.0:1789760706.595822:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685034570976 time 685034310560 state 0x1 ref 1 00000100:00100000:1.0:1789760706.595900:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685034570976 time 685061188694 state 0x0 ref 1 00000100:00100000:1.0:1789760706.595911:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515599872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.595962:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19316736, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.595971:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.595987:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760706.595991:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.595997:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.596012:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.635616:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972033 is committed 00002000:00010000:1.0:1789760706.635619:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19316736 -> 19320832 00010000:00010000:1.0:1789760706.635634:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.635647:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.635657:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.635664:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760706.635669:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.635678:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.635697:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.635731:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d35e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.635895:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515599872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39980us (40321us total) trans 4294972033 rc 0/0 00000100:00100000:1.0:1789760706.635909:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4745 00000100:00100000:1.0:1789760706.635920:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685034570976 time 685061188694 state 0x2 ref 0 00000100:00100000:0.0:1789760706.640431:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760706.640523:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600000 00000100:00100000:2.0:1789760706.640532:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600000 00000100:00100000:2.0:1789760706.640583:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685034570976 time 685061188694 state 0x0 ref 1 00000100:00100000:2.0:1789760706.640597:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685061449110 time 685061188694 state 0x1 ref 1 00000100:00100000:2.0:1789760706.640658:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685061449110 time 685105946324 state 0x0 ref 1 00000100:00100000:2.0:1789760706.640671:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515600000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760706.640875:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19320832, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:2.0:1789760706.640884:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d365 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.640900:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.640905:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.640911:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.640930:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.697548:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972034 is committed 00000100:00100000:2.0:1789760706.706101:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.706296:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600128 00000100:00100000:1.0:1789760706.706306:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600128 00000100:00100000:1.0:1789760706.706362:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685061449110 time 685105946324 state 0x0 ref 1 00000100:00100000:1.0:1789760706.706377:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685106206740 time 685105946324 state 0x1 ref 1 00002000:00010000:2.0:1789760706.753132:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19324928 -> 19329024 00010000:00010000:2.0:1789760706.753147:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.753232:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.753254:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.753263:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760706.753268:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.753331:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.753350:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.753359:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d36c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.753449:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515600128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44972us (47353us total) trans 4294972035 rc 0/0 00000100:00100000:2.0:1789760706.753459:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4747 00000100:00100000:2.0:1789760706.753468:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685106206740 time 685173544778 state 0x2 ref 0 00000100:00100000:2.0:1789760706.756697:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760706.756866:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600256 00000100:00100000:1.0:1789760706.756877:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600256 00000100:00100000:1.0:1789760706.756938:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685106206740 time 685173544778 state 0x0 ref 1 00000100:00100000:1.0:1789760706.756954:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685173805194 time 685173544778 state 0x1 ref 1 00000100:00100000:1.0:1789760706.757023:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685173805194 time 685222311488 state 0x0 ref 1 00000100:00100000:1.0:1789760706.757036:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515600256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.757083:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19329024, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9807 00010000:00010000:1.0:1789760706.757092:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d373 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.757106:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760706.757111:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.757116:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.757134:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.832196:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972036 is committed 00000100:00100000:0.0:1789760706.841258:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:2.0:1789760706.882089:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760706.906821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972037 is committed 00002000:00010000:2.0:1789760706.907054:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19333120 -> 19337216 00002000:00010000:2.0:1789760706.907059:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760706 -> 1789760707 00002000:00010000:2.0:1789760706.907061:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760706 -> 1789760707 00010000:00010000:2.0:1789760706.907074:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.907086:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.907095:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.907103:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760706.907108:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.907116:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.907142:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.907149:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d37a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.907264:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515600384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65710us (66021us total) trans 4294972037 rc 0/0 00000100:00100000:2.0:1789760706.907274:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4749 00000100:00100000:2.0:1789760706.907284:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685222571904 time 685306828489 state 0x2 ref 0 00000100:00100000:2.0:1789760706.913434:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760706.913526:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600512 00000100:00100000:0.0:1789760706.913536:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600512 00000100:00100000:0.0:1789760706.913586:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685222571904 time 685306828489 state 0x0 ref 1 00000100:00100000:0.0:1789760706.913601:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685307088905 time 685306828489 state 0x1 ref 1 00000100:00100000:0.0:1789760706.913631:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685307088905 time 685378919924 state 0x0 ref 1 00000100:00100000:0.0:1789760706.913643:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515600512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760706.913689:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19337216, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:0.0:1789760706.913698:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.913740:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760706.913745:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.913752:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.913767:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.961902:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972038 is committed 00002000:00010000:2.0:1789760706.961967:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19337216 -> 19341312 00010000:00010000:2.0:1789760706.961982:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.961996:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.962005:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.962013:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760706.962018:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.962034:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.962051:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.962059:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d381 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.962154:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515600512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48506us (48722us total) trans 4294972038 rc 0/0 00000100:00100000:2.0:1789760706.962268:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4750 00000100:00100000:2.0:1789760706.962278:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685307088905 time 685378919924 state 0x2 ref 0 00000100:00100000:1.0:1789760706.969621:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600640 00000100:00100000:1.0:1789760706.969630:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600640 00000100:00100000:1.0:1789760706.969682:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685307088905 time 685378919924 state 0x0 ref 1 00000100:00100000:1.0:1789760706.969695:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685379180340 time 685378919924 state 0x1 ref 1 00000100:00100000:1.0:1789760706.969766:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685379180340 time 685435054859 state 0x0 ref 1 00000100:00100000:1.0:1789760706.969779:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515600640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760706.969826:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19341312, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:1.0:1789760706.969835:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.969860:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760706.969865:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.969871:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.969895:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760707.007471:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19341312 -> 19345408 00010000:00010000:2.0:1789760707.007493:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.007506:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.007516:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.007523:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760707.007529:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.007538:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.007797:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.007808:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d388 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.008412:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515600640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38628us (39577us total) trans 4294972039 rc 0/0 00000100:00100000:2.0:1789760707.008423:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4751 00000100:00100000:2.0:1789760707.008433:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685379180340 time 685435054859 state 0x2 ref 0 00000001:00080000:0.0:1789760707.009648:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972039 is committed 00000100:00100000:0.0:1789760707.017346:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760707.017439:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600768 00000100:00100000:2.0:1789760707.017449:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600768 00000100:00100000:2.0:1789760707.017504:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685379180340 time 685435054859 state 0x0 ref 1 00000100:00100000:2.0:1789760707.017518:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685435315275 time 685435054859 state 0x1 ref 1 00000100:00100000:2.0:1789760707.017550:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685435315275 time 685482839331 state 0x0 ref 1 00000100:00100000:2.0:1789760707.017561:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515600768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760707.017605:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19345408, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:2.0:1789760707.017614:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d38f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.017630:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760707.017634:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.017639:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d38f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.017652:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d38f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.050491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972040 is committed 00000100:00100000:0.0:1789760707.055080:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515600896 00000100:00100000:0.0:1789760707.055090:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515600896 00000100:00100000:0.0:1789760707.055142:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685435315275 time 685482839331 state 0x0 ref 1 00000100:00100000:0.0:1789760707.055164:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685483099747 time 685482839331 state 0x1 ref 1 00000100:00100000:0.0:1789760707.055195:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685483099747 time 685520484793 state 0x0 ref 1 00000100:00100000:0.0:1789760707.055208:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515600896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760707.055263:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19349504, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:0.0:1789760707.055415:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.055431:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:0.0:1789760707.055435:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.055441:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.055460:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760707.116935:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19349504 -> 19353600 00010000:00010000:0.0:1789760707.116955:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.116968:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.116978:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.116985:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760707.116991:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.117000:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.117019:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.117027:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d396 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.117115:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515600896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61904us (62302us total) trans 4294972041 rc 0/0 00000100:00100000:0.0:1789760707.117126:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4753 00000100:00100000:0.0:1789760707.117135:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685483099747 time 685520484793 state 0x2 ref 0 00000001:00080000:0.0:1789760707.117183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972041 is committed 00000100:00100000:1.0:1789760707.126793:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760707.201493:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972042 is committed 00000100:00100000:0.0:1789760707.208628:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760707.297641:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972043 is committed 00002000:00010000:2.0:1789760707.297703:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19357696 -> 19361792 00010000:00010000:2.0:1789760707.297744:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.297757:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.297768:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.297776:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760707.297781:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.297789:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.297814:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.297821:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d3a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.305375:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760707.305649:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515601280 00000100:00100000:1.0:1789760707.305658:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515601280 00000100:00100000:1.0:1789760707.305742:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685592548484 time 685674146198 state 0x0 ref 1 00000100:00100000:1.0:1789760707.305759:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685674406614 time 685674146198 state 0x1 ref 1 00000100:00100000:1.0:1789760707.305780:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685674406614 time 685771069631 state 0x0 ref 1 00000100:00100000:1.0:1789760707.305792:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515601280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760707.305849:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19361792, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:1.0:1789760707.305858:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.305875:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760707.305879:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.305886:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.305900:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.382862:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972044 is committed 00002000:00010000:1.0:1789760707.383023:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19361792 -> 19365888 00010000:00010000:1.0:1789760707.383040:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.383053:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.383067:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.383075:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760707.383080:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.383089:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.383108:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.383115:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d3ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.383216:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515601280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 77421us (77846us total) trans 4294972044 rc 0/0 00000100:00100000:1.0:1789760707.383226:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4756 00000100:00100000:1.0:1789760707.383235:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685674406614 time 685771069631 state 0x2 ref 0 00000100:00100000:2.0:1789760707.386795:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760707.386868:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515601408 00000100:00100000:1.0:1789760707.386877:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515601408 00000100:00100000:1.0:1789760707.386982:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685674406614 time 685771069631 state 0x0 ref 1 00000100:00100000:1.0:1789760707.386997:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685771330047 time 685771069631 state 0x1 ref 1 00000100:00100000:1.0:1789760707.387053:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685771330047 time 685852341840 state 0x0 ref 1 00000100:00100000:1.0:1789760707.387065:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515601408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760707.387108:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19365888, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:1.0:1789760707.387117:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d3b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.387132:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:1.0:1789760707.387137:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.387142:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d3b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.387165:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d3b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760707.436499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972045 is committed 00000100:00100000:0.0:1789760707.448769:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760707.449077:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515601536 00000100:00100000:2.0:1789760707.449087:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515601536 00000100:00100000:2.0:1789760707.449141:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685771330047 time 685852341840 state 0x0 ref 1 00000100:00100000:2.0:1789760707.449168:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685852602256 time 685852341840 state 0x1 ref 1 00000100:00100000:2.0:1789760707.449195:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685852602256 time 685914484371 state 0x0 ref 1 00000100:00100000:2.0:1789760707.449207:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515601536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760707.449262:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19369984, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:2.0:1789760707.449313:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d3b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.449330:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760707.449334:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.449340:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d3b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.449360:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d3b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.517487:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972046 is committed 00000100:00100000:0.0:1789760707.526865:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760707.527221:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515601664 00000100:00100000:2.0:1789760707.527231:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515601664 00000100:00100000:2.0:1789760707.527295:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685852602256 time 685914484371 state 0x0 ref 1 00000100:00100000:2.0:1789760707.527310:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685914744787 time 685914484371 state 0x1 ref 1 00000100:00100000:2.0:1789760707.527448:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 685914744787 time 685992735204 state 0x0 ref 1 00000100:00100000:2.0:1789760707.527462:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515601664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760707.527510:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19374080, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:2.0:1789760707.527519:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d3c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.527535:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760707.527539:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.527546:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d3c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.527559:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d3c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760707.598634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972047 is committed 00000100:00100000:0.0:1789760707.604703:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760707.640387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972048 is committed 00000100:00100000:0.0:1789760707.644417:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760707.704297:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972049 is committed 00002000:00010000:0.0:1789760707.704538:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19382272 -> 19386368 00010000:00010000:0.0:1789760707.704553:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.704565:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.704575:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.704582:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760707.704588:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.704597:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.704617:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.704625:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d3ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.704756:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515601920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60073us (60343us total) trans 4294972049 rc 0/0 00000100:00100000:0.0:1789760707.704766:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4761 00000100:00100000:0.0:1789760707.704775:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686070817783 time 686109956057 state 0x2 ref 0 00000100:00100000:0.0:1789760707.712538:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760707.712674:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515602048 00000100:00100000:1.0:1789760707.712683:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515602048 00000100:00100000:1.0:1789760707.712771:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686070817783 time 686109956057 state 0x0 ref 1 00000100:00100000:1.0:1789760707.712787:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686110216473 time 686109956057 state 0x1 ref 1 00000100:00100000:1.0:1789760707.712815:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686110216473 time 686178104203 state 0x0 ref 1 00000100:00100000:1.0:1789760707.712827:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515602048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760707.712871:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19386368, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:1.0:1789760707.712889:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.712905:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760707.712910:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.712915:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.712928:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.780691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972050 is committed 00002000:00010000:0.0:1789760707.780869:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19386368 -> 19390464 00010000:00010000:0.0:1789760707.780895:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.780906:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.780915:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.780926:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760707.780931:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.780939:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.780965:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.780973:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d3d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.781058:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515602048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 68228us (68525us total) trans 4294972050 rc 0/0 00000100:00100000:0.0:1789760707.781067:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4762 00000100:00100000:0.0:1789760707.781076:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686110216473 time 686178104203 state 0x2 ref 0 00000100:00100000:1.0:1789760707.792763:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:0.0:1789760707.866791:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19390464 -> 19394560 00010000:00010000:0.0:1789760707.866811:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.866823:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.866839:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.866846:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760707.866852:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.866861:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.866881:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.866888:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d3dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.866984:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515602176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73940us (74271us total) trans 4294972051 rc 0/0 00000100:00100000:0.0:1789760707.866995:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4763 00000100:00100000:0.0:1789760707.867005:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686178364619 time 686258319352 state 0x2 ref 0 00000001:00080000:2.0:1789760707.867317:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972051 is committed 00000100:00100000:1.0:1789760707.872591:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760707.873763:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515602304 00000100:00100000:2.0:1789760707.873774:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515602304 00000100:00100000:2.0:1789760707.873832:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686178364619 time 686258319352 state 0x0 ref 1 00000100:00100000:2.0:1789760707.873846:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686258579768 time 686258319352 state 0x1 ref 1 00000100:00100000:2.0:1789760707.873872:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686258579768 time 686339161816 state 0x0 ref 1 00000100:00100000:2.0:1789760707.873886:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515602304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760707.873932:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19394560, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9807 00010000:00010000:2.0:1789760707.873940:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.873955:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760707.873969:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.873976:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.873990:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.905357:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760707.968774:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19394560 -> 19398656 00002000:00010000:2.0:1789760707.968783:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760707 -> 1789760708 00002000:00010000:2.0:1789760707.968785:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760707 -> 1789760708 00010000:00010000:2.0:1789760707.968799:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.968817:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.968826:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.968834:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760707.968839:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.968849:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.968868:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.968876:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.968974:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515602304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 95086us (96386us total) trans 4294972052 rc 0/0 00000100:00100000:2.0:1789760707.968985:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4764 00000100:00100000:2.0:1789760707.968995:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686258579768 time 686339161816 state 0x2 ref 0 00000100:00100000:0.0:1789760707.972813:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760708.044443:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972053 is committed 00002000:00010000:2.0:1789760708.044602:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19398656 -> 19402752 00010000:00010000:2.0:1789760708.044618:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.044632:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.044642:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.044650:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760708.044655:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.044665:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.044684:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.044691:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d3ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.044817:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515602432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71732us (72009us total) trans 4294972053 rc 0/0 00000100:00100000:2.0:1789760708.044827:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4765 00000100:00100000:2.0:1789760708.044836:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686339422232 time 686438359230 state 0x2 ref 0 00000100:00100000:0.0:1789760708.050840:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760708.053327:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515602560 00000100:00100000:1.0:1789760708.053342:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515602560 00000100:00100000:1.0:1789760708.053424:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686339422232 time 686438359230 state 0x0 ref 1 00000100:00100000:1.0:1789760708.053439:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686438619646 time 686438359230 state 0x1 ref 1 00000100:00100000:1.0:1789760708.053472:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686438619646 time 686518761650 state 0x0 ref 1 00000100:00100000:1.0:1789760708.053487:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515602560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760708.053545:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19402752, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:1.0:1789760708.053555:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.053570:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760708.053574:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.053580:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.053593:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760708.054526:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760708.094964:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544405504:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760708.095037:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789760708.095561:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760708.095594:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544405504:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760708.099221:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544405632 00000100:00100000:1.0:1789760708.099229:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544405632 00000100:00100000:1.0:1789760708.099412:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1789760708.099421:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544405632:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760708.099443:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3838836736 avail 3830448128 00000100:00080000:2.0:1789760708.099789:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760708.106098:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544405632:12345-0@lo:13:osp-pre-0-0.0 Request processed in 6667us (6999us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760708.106112:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000001:00080000:0.0:1789760708.123533:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972054 is committed 00002000:00010000:2.0:1789760708.124074:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19402752 -> 19406848 00010000:00010000:2.0:1789760708.124089:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.124100:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.124108:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.124116:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760708.124121:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.124129:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.124147:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.124360:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d3f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.124454:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515602560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70966us (73619us total) trans 4294972054 rc 0/0 00000100:00100000:2.0:1789760708.124468:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4766 00000100:00100000:2.0:1789760708.124477:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686438619646 time 686518761650 state 0x2 ref 0 00000100:00100000:1.0:1789760708.129058:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760708.129114:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515602688 00000100:00100000:0.0:1789760708.129125:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515602688 00000100:00100000:0.0:1789760708.129864:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686438619646 time 686518761650 state 0x0 ref 1 00000100:00100000:0.0:1789760708.130049:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686519022066 time 686518761650 state 0x1 ref 1 00000100:00100000:2.0:1789760708.130142:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686519022066 time 686595428973 state 0x0 ref 1 00000100:00100000:2.0:1789760708.130171:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515602688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760708.130226:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19406848, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:2.0:1789760708.130249:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.130266:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760708.130278:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.130285:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.130298:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.203024:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972055 is committed 00002000:00010000:0.0:1789760708.203654:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19406848 -> 19410944 00010000:00010000:0.0:1789760708.203679:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.203693:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.203703:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.203746:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.203751:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.203761:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.203787:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.203795:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d3f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.203878:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515602688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73710us (74824us total) trans 4294972055 rc 0/0 00000100:00100000:0.0:1789760708.203888:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4767 00000100:00100000:0.0:1789760708.203897:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686519022066 time 686595428973 state 0x2 ref 0 00000100:00100000:0.0:1789760708.208014:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760708.208368:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515602816 00000100:00100000:1.0:1789760708.208378:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515602816 00000100:00100000:1.0:1789760708.208430:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686519022066 time 686595428973 state 0x0 ref 1 00000100:00100000:1.0:1789760708.208445:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686595689389 time 686595428973 state 0x1 ref 1 00000100:00100000:1.0:1789760708.208478:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686595689389 time 686673767206 state 0x0 ref 1 00000100:00100000:1.0:1789760708.208493:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515602816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760708.208547:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19410944, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:1.0:1789760708.208555:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.208570:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:1.0:1789760708.208575:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.208580:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.208592:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.292028:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972056 is committed 00002000:00010000:0.0:1789760708.292155:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19410944 -> 19415040 00010000:00010000:0.0:1789760708.292170:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.292190:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.292199:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.292207:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.292212:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.292225:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.292253:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.292260:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1d3ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.292355:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515602816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 83859us (84345us total) trans 4294972056 rc 0/0 00000100:00100000:0.0:1789760708.292365:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4768 00000100:00100000:0.0:1789760708.292374:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686595689389 time 686673767206 state 0x2 ref 0 00000100:00100000:1.0:1789760708.299606:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760708.369694:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19415040 -> 19419136 00010000:00010000:1.0:1789760708.369728:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.369741:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.369750:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.369756:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760708.369762:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.369771:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.369789:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.369797:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d406 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.369927:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515602944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70059us (70326us total) trans 4294972057 rc 0/0 00000100:00100000:1.0:1789760708.369938:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4769 00000100:00100000:1.0:1789760708.369947:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686674027622 time 686765141436 state 0x2 ref 0 00000100:00100000:1.0:1789760708.380550:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00002000:00010000:1.0:1789760708.431621:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19419136 -> 19423232 00010000:00010000:1.0:1789760708.431640:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.431652:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.431661:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.431673:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760708.431677:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.431686:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.431704:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.431727:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d40d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.431857:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515603072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51084us (51310us total) trans 4294972058 rc 0/0 00000100:00100000:1.0:1789760708.431877:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4770 00000100:00100000:1.0:1789760708.431888:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686765401852 time 686846046145 state 0x2 ref 0 00000100:00100000:0.0:1789760708.437149:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00010000:00010000:3.0:1789760708.437443:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.437456:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.496643:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972059 is committed 00002000:00010000:0.0:1789760708.496758:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19423232 -> 19427328 00010000:00010000:0.0:1789760708.496771:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.496784:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.496793:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.496804:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.496809:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.496818:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.496909:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.496917:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d414 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.497061:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515603200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59694us (59915us total) trans 4294972059 rc 0/0 00000100:00100000:0.0:1789760708.497072:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4771 00000100:00100000:0.0:1789760708.497081:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686846306561 time 686902640028 state 0x2 ref 0 00000100:00100000:0.0:1789760708.512410:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760708.512496:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515603328 00000100:00100000:2.0:1789760708.512507:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515603328 00000100:00100000:2.0:1789760708.512556:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686846306561 time 686902640028 state 0x0 ref 1 00000100:00100000:2.0:1789760708.512572:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686902900444 time 686902640028 state 0x1 ref 1 00000100:00100000:2.0:1789760708.512609:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686902900444 time 686977897241 state 0x0 ref 1 00000100:00100000:2.0:1789760708.512622:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515603328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760708.512671:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19427328, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:2.0:1789760708.512680:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.512694:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760708.512698:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.512731:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.512746:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.576129:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972060 is committed 00002000:00010000:0.0:1789760708.576664:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19427328 -> 19431424 00010000:00010000:0.0:1789760708.576680:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.576693:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.576736:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.576745:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.576749:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.576758:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.576779:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.576787:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d41b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.576868:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515603328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64244us (64463us total) trans 4294972060 rc 0/0 00000100:00100000:0.0:1789760708.576878:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4772 00000100:00100000:0.0:1789760708.576887:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686902900444 time 686977897241 state 0x2 ref 0 00000100:00100000:1.0:1789760708.583839:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760708.584437:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515603456 00000100:00100000:0.0:1789760708.584447:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515603456 00000100:00100000:0.0:1789760708.584560:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686902900444 time 686977897241 state 0x0 ref 1 00000100:00100000:0.0:1789760708.584577:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686978157657 time 686977897241 state 0x1 ref 1 00000100:00100000:0.0:1789760708.584644:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686978157657 time 687049932859 state 0x0 ref 1 00000100:00100000:0.0:1789760708.584657:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515603456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760708.584737:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19431424, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:0.0:1789760708.584745:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.584761:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760708.584766:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.584772:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.584785:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760708.624766:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19431424 -> 19435520 00010000:00010000:0.0:1789760708.624786:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.624796:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.624804:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.624810:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.624815:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.624824:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.624842:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.624850:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d422 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.624934:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515603456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40275us (41099us total) trans 4294972061 rc 0/0 00000100:00100000:0.0:1789760708.624944:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4773 00000100:00100000:0.0:1789760708.624953:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686978157657 time 687049932859 state 0x2 ref 0 00000001:00080000:3.0:1789760708.625522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972061 is committed 00000100:00100000:1.0:1789760708.635181:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760708.635444:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515603584 00000100:00100000:0.0:1789760708.635455:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515603584 00000100:00100000:0.0:1789760708.635510:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 686978157657 time 687049932859 state 0x0 ref 1 00000100:00100000:0.0:1789760708.635526:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687050193275 time 687049932859 state 0x1 ref 1 00000100:00100000:0.0:1789760708.635557:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687050193275 time 687100846023 state 0x0 ref 1 00000100:00100000:0.0:1789760708.635569:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515603584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760708.635618:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19435520, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:0.0:1789760708.635627:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.635643:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760708.635648:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.635654:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.635668:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.664083:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972062 is committed 00002000:00010000:0.0:1789760708.664157:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19435520 -> 19439616 00010000:00010000:0.0:1789760708.664172:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.664184:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.664192:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.664200:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.664204:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.664211:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.664239:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.664247:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d429 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.664454:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515603584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28881us (29277us total) trans 4294972062 rc 0/0 00000100:00100000:0.0:1789760708.664465:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4774 00000100:00100000:0.0:1789760708.664474:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687050193275 time 687100846023 state 0x2 ref 0 00000100:00100000:1.0:1789760708.669437:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760708.669535:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515603712 00000100:00100000:3.0:1789760708.669544:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515603712 00000100:00100000:3.0:1789760708.669593:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687050193275 time 687100846023 state 0x0 ref 1 00000100:00100000:3.0:1789760708.669607:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687101106439 time 687100846023 state 0x1 ref 1 00000100:00100000:3.0:1789760708.669667:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687101106439 time 687134956712 state 0x0 ref 1 00000100:00100000:3.0:1789760708.669679:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515603712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760708.669745:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19439616, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:3.0:1789760708.669754:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.669769:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.669774:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.669781:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.669793:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.704701:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972063 is committed 00002000:00010000:1.0:1789760708.704854:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19439616 -> 19443712 00010000:00010000:1.0:1789760708.705013:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.705028:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.705038:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.705045:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760708.705050:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.705058:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.705077:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.705085:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d430 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.705184:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515603712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35502us (35752us total) trans 4294972063 rc 0/0 00000100:00100000:1.0:1789760708.705194:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4775 00000100:00100000:1.0:1789760708.705203:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687101106439 time 687134956712 state 0x2 ref 0 00000100:00100000:1.0:1789760708.709041:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760708.709120:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515603840 00000100:00100000:3.0:1789760708.709130:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515603840 00000100:00100000:3.0:1789760708.709190:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687101106439 time 687134956712 state 0x0 ref 1 00000100:00100000:3.0:1789760708.709205:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687135217128 time 687134956712 state 0x1 ref 1 00000100:00100000:3.0:1789760708.709225:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687135217128 time 687174515021 state 0x0 ref 1 00000100:00100000:3.0:1789760708.709248:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515603840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760708.709425:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19443712, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:3.0:1789760708.709434:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.709450:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.709454:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.709461:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.709473:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.745387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972064 is committed 00002000:00010000:3.0:1789760708.745501:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19443712 -> 19447808 00010000:00010000:3.0:1789760708.745517:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.745537:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.745546:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.745553:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760708.745558:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.745567:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.745587:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.745595:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d437 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.745686:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515603840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36435us (36649us total) trans 4294972064 rc 0/0 00000100:00100000:3.0:1789760708.745696:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4776 00000100:00100000:3.0:1789760708.745740:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687135217128 time 687174515021 state 0x2 ref 0 00000100:00100000:2.0:1789760708.749987:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760708.750485:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515603968 00000100:00100000:3.0:1789760708.750496:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515603968 00000100:00100000:3.0:1789760708.750547:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687135217128 time 687174515021 state 0x0 ref 1 00000100:00100000:3.0:1789760708.750562:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687174775437 time 687174515021 state 0x1 ref 1 00000100:00100000:3.0:1789760708.750595:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687174775437 time 687215884063 state 0x0 ref 1 00000100:00100000:3.0:1789760708.750607:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515603968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760708.750654:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19447808, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:3.0:1789760708.750663:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.750679:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.750684:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.750690:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.750704:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.782971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972065 is committed 00002000:00010000:3.0:1789760708.783162:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19447808 -> 19451904 00010000:00010000:3.0:1789760708.783178:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.783189:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.783202:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.783209:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760708.783214:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.783222:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.783250:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.783372:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d43e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.783469:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515603968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32860us (33487us total) trans 4294972065 rc 0/0 00000100:00100000:3.0:1789760708.783479:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4777 00000100:00100000:3.0:1789760708.783489:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687174775437 time 687215884063 state 0x2 ref 0 00000100:00100000:0.0:1789760708.786901:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760708.787025:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604096 00000100:00100000:2.0:1789760708.787034:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604096 00000100:00100000:2.0:1789760708.787099:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687174775437 time 687215884063 state 0x0 ref 1 00000100:00100000:2.0:1789760708.787114:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687216144479 time 687215884063 state 0x1 ref 1 00000100:00100000:2.0:1789760708.787155:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687216144479 time 687252434658 state 0x0 ref 1 00000100:00100000:2.0:1789760708.787166:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760708.787211:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19451904, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:2.0:1789760708.787221:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.787245:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.787250:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.787256:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.787391:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.830576:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972066 is committed 00002000:00010000:2.0:1789760708.830803:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19451904 -> 19456000 00010000:00010000:2.0:1789760708.830819:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.830830:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.830839:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.830847:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760708.830852:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.830860:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.830894:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.830902:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1d445 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.830996:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43825us (44099us total) trans 4294972066 rc 0/0 00000100:00100000:2.0:1789760708.831006:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4778 00000100:00100000:2.0:1789760708.831014:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687216144479 time 687252434658 state 0x2 ref 0 00000100:00100000:2.0:1789760708.838579:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760708.840154:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604224 00000100:00100000:3.0:1789760708.840167:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604224 00000100:00100000:3.0:1789760708.840309:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687216144479 time 687252434658 state 0x0 ref 1 00000100:00100000:3.0:1789760708.840328:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687252695074 time 687252434658 state 0x1 ref 1 00000100:00100000:3.0:1789760708.840362:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687252695074 time 687305650976 state 0x0 ref 1 00000100:00100000:3.0:1789760708.840375:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515604224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760708.840423:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19456000, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9807 00010000:00010000:3.0:1789760708.840432:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.840447:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760708.840452:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.840458:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.840471:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760708.913470:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19456000 -> 19460096 00002000:00010000:0.0:1789760708.913479:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760708 -> 1789760709 00002000:00010000:0.0:1789760708.913481:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760708 -> 1789760709 00010000:00010000:0.0:1789760708.913494:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.913506:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.913515:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.913539:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760708.913544:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.913553:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.913571:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.913578:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d44c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19456000->19460095] (req 19456000->19460095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.913666:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515604224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73287us (75091us total) trans 4294972067 rc 0/0 00000100:00100000:0.0:1789760708.913676:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4779 00000100:00100000:0.0:1789760708.913685:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687252695074 time 687305650976 state 0x2 ref 0 00000001:00080000:0.0:1789760708.913768:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972067 is committed 00000100:00100000:0.0:1789760708.918466:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760708.918542:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604352 00000100:00100000:1.0:1789760708.918552:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604352 00000100:00100000:1.0:1789760708.918604:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687252695074 time 687305650976 state 0x0 ref 1 00000100:00100000:1.0:1789760708.918620:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687305911392 time 687305650976 state 0x1 ref 1 00000100:00100000:1.0:1789760708.918650:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687305911392 time 687383939268 state 0x0 ref 1 00000100:00100000:1.0:1789760708.918663:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515604352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760708.919167:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19460096, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:0.0:1789760708.919179:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.919195:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760708.919200:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.919207:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.919222:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.925853:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760709.009654:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972068 is committed 00002000:00010000:0.0:1789760709.010234:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19460096 -> 19464192 00010000:00010000:0.0:1789760709.010250:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.010272:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.010281:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.010288:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760709.010293:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.010302:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.010321:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.010335:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d453 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.010429:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515604352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 91763us (91967us total) trans 4294972068 rc 0/0 00000100:00100000:0.0:1789760709.010440:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4780 00000100:00100000:0.0:1789760709.010450:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687305911392 time 687383939268 state 0x2 ref 0 00000100:00100000:1.0:1789760709.015351:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.015398:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604480 00000100:00100000:3.0:1789760709.015408:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604480 00000100:00100000:3.0:1789760709.015459:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687305911392 time 687383939268 state 0x0 ref 1 00000100:00100000:3.0:1789760709.015474:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687384199684 time 687383939268 state 0x1 ref 1 00000100:00100000:3.0:1789760709.015504:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687384199684 time 687480793226 state 0x0 ref 1 00000100:00100000:3.0:1789760709.015515:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.015629:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19464192, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.015639:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.015653:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760709.015657:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.015662:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.015673:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.063612:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972069 is committed 00002000:00010000:2.0:1789760709.063858:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19464192 -> 19468288 00010000:00010000:2.0:1789760709.063882:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.063894:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.063902:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.063908:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760709.063913:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.063926:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.063943:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.063960:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d45a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.064046:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48527us (48698us total) trans 4294972069 rc 0/0 00000100:00100000:2.0:1789760709.064056:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4781 00000100:00100000:2.0:1789760709.064065:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687384199684 time 687480793226 state 0x2 ref 0 00000100:00100000:1.0:1789760709.073473:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.073558:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604608 00000100:00100000:3.0:1789760709.073568:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604608 00000100:00100000:3.0:1789760709.073623:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687384199684 time 687480793226 state 0x0 ref 1 00000100:00100000:3.0:1789760709.073637:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687481053642 time 687480793226 state 0x1 ref 1 00000100:00100000:3.0:1789760709.073668:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687481053642 time 687538957328 state 0x0 ref 1 00000100:00100000:3.0:1789760709.073680:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515604608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.073762:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19468288, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.073772:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.073793:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760709.073798:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.073804:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.073818:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760709.177165:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19468288 -> 19472384 00010000:00010000:3.0:1789760709.177186:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.177198:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.177206:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.177214:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.177219:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.177238:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.177258:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.177375:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d461 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.177470:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515604608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 103786us (104001us total) trans 4294972070 rc 0/0 00000100:00100000:3.0:1789760709.177482:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4782 00000100:00100000:3.0:1789760709.177492:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687481053642 time 687538957328 state 0x2 ref 0 00000001:00080000:2.0:1789760709.177635:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972070 is committed 00000100:00100000:2.0:1789760709.183172:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.183487:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604736 00000100:00100000:3.0:1789760709.183501:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604736 00000100:00100000:3.0:1789760709.183573:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687481053642 time 687538957328 state 0x0 ref 1 00000100:00100000:3.0:1789760709.183588:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687539217744 time 687538957328 state 0x1 ref 1 00000100:00100000:3.0:1789760709.183618:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687539217744 time 687648907251 state 0x0 ref 1 00000100:00100000:3.0:1789760709.183631:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.183683:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19472384, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.183692:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.183745:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760709.183750:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.183758:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.183778:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.292098:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972071 is committed 00002000:00010000:3.0:1789760709.292151:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19472384 -> 19476480 00010000:00010000:3.0:1789760709.292167:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.292180:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.292190:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.292197:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.292202:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.292210:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.292239:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.292247:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d468 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.292353:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 108721us (109186us total) trans 4294972071 rc 0/0 00000100:00100000:3.0:1789760709.292363:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4783 00000100:00100000:3.0:1789760709.292373:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687539217744 time 687648907251 state 0x2 ref 0 00000100:00100000:1.0:1789760709.298982:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760709.299093:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604864 00000100:00100000:0.0:1789760709.299102:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604864 00000100:00100000:0.0:1789760709.299264:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687539217744 time 687648907251 state 0x0 ref 1 00000100:00100000:0.0:1789760709.300896:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687649167667 time 687648907251 state 0x1 ref 1 00000100:00100000:0.0:1789760709.300952:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687649167667 time 687766237183 state 0x0 ref 1 00000100:00100000:0.0:1789760709.300975:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760709.301026:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19476480, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:0.0:1789760709.301035:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.301052:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760709.301057:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.301063:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.301083:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.345022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972072 is committed 00002000:00010000:1.0:1789760709.345085:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19476480 -> 19480576 00010000:00010000:1.0:1789760709.345101:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.345114:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.345124:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.345130:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760709.345135:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.345145:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.345175:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.345183:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d46f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.345275:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515604864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44306us (46297us total) trans 4294972072 rc 0/0 00000100:00100000:1.0:1789760709.345284:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4784 00000100:00100000:1.0:1789760709.345295:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687649167667 time 687766237183 state 0x2 ref 0 00000100:00100000:1.0:1789760709.350493:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760709.350690:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515604992 00000100:00100000:0.0:1789760709.350700:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515604992 00000100:00100000:0.0:1789760709.350775:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687649167667 time 687766237183 state 0x0 ref 1 00000100:00100000:0.0:1789760709.350789:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687766497599 time 687766237183 state 0x1 ref 1 00000100:00100000:0.0:1789760709.350813:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687766497599 time 687816102588 state 0x0 ref 1 00000100:00100000:0.0:1789760709.350825:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515604992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760709.350868:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19480576, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:0.0:1789760709.350877:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.350892:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760709.350897:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.350904:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.350919:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760709.385076:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19480576 -> 19484672 00010000:00010000:0.0:1789760709.385096:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.385111:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.385112:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972073 is committed 00010000:00010000:0.0:1789760709.385125:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.385134:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760709.385139:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.385149:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.385168:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.385175:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d476 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.385271:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515604992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34443us (34782us total) trans 4294972073 rc 0/0 00000100:00100000:0.0:1789760709.385699:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4785 00000100:00100000:0.0:1789760709.386063:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687766497599 time 687816102588 state 0x2 ref 0 00000100:00100000:2.0:1789760709.397382:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760709.397783:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605120 00000100:00100000:0.0:1789760709.397793:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605120 00000100:00100000:0.0:1789760709.397855:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687766497599 time 687816102588 state 0x0 ref 1 00000100:00100000:0.0:1789760709.397880:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687816363004 time 687816102588 state 0x1 ref 1 00000100:00100000:0.0:1789760709.397909:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687816363004 time 687863192791 state 0x0 ref 1 00000100:00100000:0.0:1789760709.397923:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515605120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760709.397978:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19484672, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:0.0:1789760709.397987:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.398002:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760709.398006:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.398012:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.398031:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.436440:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972074 is committed 00002000:00010000:3.0:1789760709.436691:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19484672 -> 19488768 00010000:00010000:3.0:1789760709.436737:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.436750:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.436759:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.436767:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.436772:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.436781:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.436800:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.436813:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d47d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.436901:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515605120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38976us (39523us total) trans 4294972074 rc 0/0 00000100:00100000:3.0:1789760709.436911:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4786 00000100:00100000:3.0:1789760709.436921:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687816363004 time 687863192791 state 0x2 ref 0 00000100:00100000:1.0:1789760709.440970:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760709.441333:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605248 00000100:00100000:2.0:1789760709.441344:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605248 00000100:00100000:2.0:1789760709.441396:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687816363004 time 687863192791 state 0x0 ref 1 00000100:00100000:2.0:1789760709.441412:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687863453207 time 687863192791 state 0x1 ref 1 00000100:00100000:2.0:1789760709.441443:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687863453207 time 687906731783 state 0x0 ref 1 00000100:00100000:2.0:1789760709.441459:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515605248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760709.441509:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19488768, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:2.0:1789760709.441519:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.441536:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760709.441541:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.441548:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.441563:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.505530:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972075 is committed 00002000:00010000:3.0:1789760709.505659:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19488768 -> 19492864 00010000:00010000:3.0:1789760709.505674:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.505687:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.505696:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.505702:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.505738:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.505748:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.505772:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.505779:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d484 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.505876:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515605248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64415us (64912us total) trans 4294972075 rc 0/0 00000100:00100000:3.0:1789760709.505886:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4787 00000100:00100000:3.0:1789760709.505896:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687863453207 time 687906731783 state 0x2 ref 0 00000100:00100000:1.0:1789760709.511492:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.511563:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605376 00000100:00100000:3.0:1789760709.511572:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605376 00000100:00100000:3.0:1789760709.511621:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687863453207 time 687906731783 state 0x0 ref 1 00000100:00100000:3.0:1789760709.511635:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687906992199 time 687906731783 state 0x1 ref 1 00000100:00100000:3.0:1789760709.511666:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687906992199 time 687976955601 state 0x0 ref 1 00000100:00100000:3.0:1789760709.511677:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515605376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.511752:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19492864, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.511761:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.511775:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760709.511780:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.511786:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.511799:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.529203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972076 is committed 00002000:00010000:3.0:1789760709.529345:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19492864 -> 19496960 00010000:00010000:3.0:1789760709.529360:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.529373:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.529383:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.529391:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.529397:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.529405:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.529426:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.529434:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d48b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.529525:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515605376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 17844us (18037us total) trans 4294972076 rc 0/0 00000100:00100000:3.0:1789760709.529536:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4788 00000100:00100000:3.0:1789760709.529545:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687906992199 time 687976955601 state 0x2 ref 0 00000100:00100000:2.0:1789760709.541599:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.542821:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605504 00000100:00100000:3.0:1789760709.542833:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605504 00000100:00100000:3.0:1789760709.542889:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687906992199 time 687976955601 state 0x0 ref 1 00000100:00100000:3.0:1789760709.542903:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687977216017 time 687976955601 state 0x1 ref 1 00000100:00100000:3.0:1789760709.542961:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687977216017 time 688008250046 state 0x0 ref 1 00000100:00100000:3.0:1789760709.542973:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515605504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.543019:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19496960, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.543028:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.543045:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760709.543049:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.543056:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.543070:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.578053:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972077 is committed 00002000:00010000:1.0:1789760709.578157:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19496960 -> 19501056 00010000:00010000:1.0:1789760709.578173:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.578186:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.578197:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.578205:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760709.578210:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.578229:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.578251:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.578265:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d492 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.578350:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515605504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35374us (36755us total) trans 4294972077 rc 0/0 00000100:00100000:1.0:1789760709.578359:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4789 00000100:00100000:1.0:1789760709.578368:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687977216017 time 688008250046 state 0x2 ref 0 00000100:00100000:2.0:1789760709.583588:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760709.583667:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605632 00000100:00100000:1.0:1789760709.583677:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605632 00000100:00100000:1.0:1789760709.583778:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 687977216017 time 688008250046 state 0x0 ref 1 00000100:00100000:1.0:1789760709.583795:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688008510462 time 688008250046 state 0x1 ref 1 00000100:00100000:1.0:1789760709.583821:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688008510462 time 688049111035 state 0x0 ref 1 00000100:00100000:1.0:1789760709.583837:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515605632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760709.583883:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19501056, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:1.0:1789760709.583892:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.583907:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.583911:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.583917:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.583929:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.610816:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972078 is committed 00002000:00010000:2.0:1789760709.610933:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19501056 -> 19505152 00010000:00010000:2.0:1789760709.610947:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.610960:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.610970:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.610978:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760709.610983:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.610992:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.611011:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.611018:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d499 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.611132:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515605632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27293us (27549us total) trans 4294972078 rc 0/0 00000100:00100000:2.0:1789760709.611152:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4790 00000100:00100000:2.0:1789760709.611161:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688008510462 time 688049111035 state 0x2 ref 0 00000100:00100000:2.0:1789760709.615453:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760709.615554:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605760 00000100:00100000:0.0:1789760709.615564:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605760 00000100:00100000:0.0:1789760709.615611:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688008510462 time 688049111035 state 0x0 ref 1 00000100:00100000:0.0:1789760709.615626:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688049371451 time 688049111035 state 0x1 ref 1 00000100:00100000:0.0:1789760709.615656:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688049371451 time 688080945383 state 0x0 ref 1 00000100:00100000:0.0:1789760709.615669:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515605760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760709.615902:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19505152, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:1.0:1789760709.615913:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.615928:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.615933:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.615938:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.615953:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760709.643501:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19505152 -> 19509248 00010000:00010000:2.0:1789760709.643520:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.643535:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.643550:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.643558:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760709.643563:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.643572:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.643591:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.643599:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d4a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.643683:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515605760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28011us (28460us total) trans 4294972079 rc 0/0 00000100:00100000:2.0:1789760709.643693:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4791 00000100:00100000:2.0:1789760709.643703:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688049371451 time 688080945383 state 0x2 ref 0 00000001:00080000:3.0:1789760709.644094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972079 is committed 00000100:00100000:0.0:1789760709.648926:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760709.650619:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515605888 00000100:00100000:1.0:1789760709.650631:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515605888 00000100:00100000:1.0:1789760709.650692:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688049371451 time 688080945383 state 0x0 ref 1 00000100:00100000:1.0:1789760709.650788:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688081205799 time 688080945383 state 0x1 ref 1 00000100:00100000:1.0:1789760709.651066:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688081205799 time 688116352829 state 0x0 ref 1 00000100:00100000:1.0:1789760709.651082:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515605888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760709.651133:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19509248, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:1.0:1789760709.651448:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.651465:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.651470:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.651477:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.651491:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760709.697432:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19509248 -> 19513344 00010000:00010000:0.0:1789760709.697451:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.697464:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.697473:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.697481:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760709.697485:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.697495:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.697514:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.697521:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d4a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.697616:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515605888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46534us (48693us total) trans 4294972080 rc 0/0 00000100:00100000:0.0:1789760709.697626:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4792 00000100:00100000:0.0:1789760709.697636:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688081205799 time 688116352829 state 0x2 ref 0 00000001:00080000:3.0:1789760709.698543:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972080 is committed 00000100:00100000:1.0:1789760709.713512:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.713587:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606016 00000100:00100000:3.0:1789760709.713597:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606016 00000100:00100000:3.0:1789760709.713648:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688081205799 time 688116352829 state 0x0 ref 1 00000100:00100000:3.0:1789760709.713662:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688116613245 time 688116352829 state 0x1 ref 1 00000100:00100000:3.0:1789760709.713690:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688116613245 time 688178979282 state 0x0 ref 1 00000100:00100000:3.0:1789760709.713702:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515606016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.713777:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19513344, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.713787:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.713801:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760709.713806:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.713812:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.713825:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.819353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972081 is committed 00002000:00010000:3.0:1789760709.819583:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19513344 -> 19517440 00010000:00010000:3.0:1789760709.819597:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.819620:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.819630:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.819638:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.819643:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.819652:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.819671:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.819678:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.819788:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515606016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 106083us (106280us total) trans 4294972081 rc 0/0 00000100:00100000:3.0:1789760709.819799:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4793 00000100:00100000:3.0:1789760709.819807:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688116613245 time 688178979282 state 0x2 ref 0 00000100:00100000:1.0:1789760709.827362:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.827600:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606144 00000100:00100000:3.0:1789760709.827618:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606144 00000100:00100000:3.0:1789760709.827667:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688116613245 time 688178979282 state 0x0 ref 1 00000100:00100000:3.0:1789760709.827682:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688179239698 time 688178979282 state 0x1 ref 1 00000100:00100000:3.0:1789760709.827732:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688179239698 time 688292997782 state 0x0 ref 1 00000100:00100000:3.0:1789760709.827745:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515606144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.827789:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19517440, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9807 00010000:00010000:3.0:1789760709.827797:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.827812:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760709.827816:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.827823:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.827836:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760709.898894:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19517440 -> 19521536 00002000:00010000:3.0:1789760709.898902:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760709 -> 1789760710 00002000:00010000:3.0:1789760709.898906:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760709 -> 1789760710 00010000:00010000:3.0:1789760709.898919:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.898939:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.898949:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.898956:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.898961:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.898970:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.898988:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.898997:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.899085:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515606144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71337us (71727us total) trans 4294972082 rc 0/0 00000100:00100000:3.0:1789760709.899095:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4794 00000100:00100000:3.0:1789760709.899105:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688179239698 time 688292997782 state 0x2 ref 0 00000001:00080000:2.0:1789760709.900543:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972082 is committed 00000100:00100000:0.0:1789760709.902234:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760709.902398:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606272 00000100:00100000:3.0:1789760709.902407:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606272 00000100:00100000:3.0:1789760709.902453:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688179239698 time 688292997782 state 0x0 ref 1 00000100:00100000:3.0:1789760709.902468:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688293258198 time 688292997782 state 0x1 ref 1 00000100:00100000:3.0:1789760709.902537:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688293258198 time 688367825574 state 0x0 ref 1 00000100:00100000:3.0:1789760709.902548:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760709.902589:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19521536, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760709.902597:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.902620:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760709.902625:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.902631:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.902645:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.933553:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972083 is committed 00002000:00010000:3.0:1789760709.933689:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19521536 -> 19525632 00010000:00010000:3.0:1789760709.933704:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.933775:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.933791:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.933798:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.933803:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.933810:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.933827:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.933833:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.933975:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31423us (31754us total) trans 4294972083 rc 0/0 00000100:00100000:3.0:1789760709.933985:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4795 00000100:00100000:3.0:1789760709.933995:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688293258198 time 688367825574 state 0x2 ref 0 00000100:00100000:1.0:1789760709.939251:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760709.939455:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606400 00000100:00100000:2.0:1789760709.939465:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606400 00000100:00100000:2.0:1789760709.939514:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688293258198 time 688367825574 state 0x0 ref 1 00000100:00100000:2.0:1789760709.939532:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688368085990 time 688367825574 state 0x1 ref 1 00000100:00100000:2.0:1789760709.939559:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688368085990 time 688404848337 state 0x0 ref 1 00000100:00100000:2.0:1789760709.939570:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515606400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760709.939803:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19525632, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760709.939812:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.939827:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.939831:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.939837:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.939852:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.950270:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760709.963167:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972084 is committed 00002000:00010000:3.0:1789760709.963511:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19525632 -> 19529728 00010000:00010000:3.0:1789760709.963525:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.963544:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.963554:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.963561:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760709.963565:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.963575:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.963594:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.963601:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d4c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19525632->19529727] (req 19525632->19529727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.963845:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515606400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24271us (24597us total) trans 4294972084 rc 0/0 00000100:00100000:3.0:1789760709.963856:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4796 00000100:00100000:3.0:1789760709.963865:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688368085990 time 688404848337 state 0x2 ref 0 00000100:00100000:1.0:1789760709.966968:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760709.967045:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606528 00000100:00100000:2.0:1789760709.967053:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606528 00000100:00100000:2.0:1789760709.967098:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688368085990 time 688404848337 state 0x0 ref 1 00000100:00100000:2.0:1789760709.967111:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688405108753 time 688404848337 state 0x1 ref 1 00000100:00100000:2.0:1789760709.967145:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688405108753 time 688432428305 state 0x0 ref 1 00000100:00100000:2.0:1789760709.967157:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760709.967204:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19529728, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760709.967213:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.967236:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.967241:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.967246:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.967266:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.000398:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972085 is committed 00002000:00010000:2.0:1789760710.000492:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19529728 -> 19533824 00010000:00010000:2.0:1789760710.000507:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.000520:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.000529:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.000537:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760710.000541:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.000551:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.000570:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.000578:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.000698:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33538us (33735us total) trans 4294972085 rc 0/0 00000100:00100000:2.0:1789760710.000793:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4797 00000100:00100000:2.0:1789760710.000802:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688405108753 time 688432428305 state 0x2 ref 0 00000100:00100000:1.0:1789760710.004048:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.004325:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606656 00000100:00100000:2.0:1789760710.004334:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606656 00000100:00100000:2.0:1789760710.004394:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688405108753 time 688432428305 state 0x0 ref 1 00000100:00100000:2.0:1789760710.004408:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688432688721 time 688432428305 state 0x1 ref 1 00000100:00100000:2.0:1789760710.004468:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688432688721 time 688469756923 state 0x0 ref 1 00000100:00100000:2.0:1789760710.004488:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.004531:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19533824, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.004539:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004554:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760710.004559:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.004565:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.004579:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760710.005178:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760710.035465:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19533824 -> 19537920 00010000:00010000:2.0:1789760710.035483:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.035497:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.035506:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.035514:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760710.035519:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.035528:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.035547:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.035555:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d4d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.035638:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515606656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31146us (31595us total) trans 4294972086 rc 0/0 00000100:00100000:2.0:1789760710.035649:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4798 00000100:00100000:2.0:1789760710.035658:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688432688721 time 688469756923 state 0x2 ref 0 00000100:00100000:1.0:1789760710.039454:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.039530:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606784 00000100:00100000:3.0:1789760710.039540:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606784 00000100:00100000:3.0:1789760710.039588:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688432688721 time 688469756923 state 0x0 ref 1 00000100:00100000:3.0:1789760710.039603:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688470017339 time 688469756923 state 0x1 ref 1 00000100:00100000:3.0:1789760710.039633:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688470017339 time 688504922190 state 0x0 ref 1 00000100:00100000:3.0:1789760710.039644:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515606784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.039689:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19537920, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.039697:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.039745:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.039750:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.039756:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.039769:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.040458:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972086 is committed 00000001:00080000:1.0:1789760710.067330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972087 is committed 00002000:00010000:0.0:1789760710.067416:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19537920 -> 19542016 00010000:00010000:0.0:1789760710.067431:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.067444:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.067455:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.067462:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.067468:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.067483:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.067503:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.067511:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19537920->19542015] (req 19537920->19542015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.067622:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515606784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27976us (28173us total) trans 4294972087 rc 0/0 00000100:00100000:0.0:1789760710.067633:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4799 00000100:00100000:0.0:1789760710.067643:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688470017339 time 688504922190 state 0x2 ref 0 00000100:00100000:1.0:1789760710.071438:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.071506:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515606912 00000100:00100000:3.0:1789760710.071515:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515606912 00000100:00100000:3.0:1789760710.071559:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688470017339 time 688504922190 state 0x0 ref 1 00000100:00100000:3.0:1789760710.071573:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688505182606 time 688504922190 state 0x1 ref 1 00000100:00100000:3.0:1789760710.071601:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688505182606 time 688536890336 state 0x0 ref 1 00000100:00100000:3.0:1789760710.071615:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.071656:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19542016, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.071664:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.071679:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.071683:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.071689:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.071703:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.097630:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972088 is committed 00002000:00010000:0.0:1789760710.097743:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19542016 -> 19546112 00010000:00010000:0.0:1789760710.097759:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.097772:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.097781:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.097788:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.097794:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.097803:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.097822:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.097829:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d4df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.097913:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515606912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26295us (26478us total) trans 4294972088 rc 0/0 00000100:00100000:0.0:1789760710.097924:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4800 00000100:00100000:0.0:1789760710.097933:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688505182606 time 688536890336 state 0x2 ref 0 00000100:00100000:0.0:1789760710.100766:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.100830:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607040 00000100:00100000:3.0:1789760710.100840:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607040 00000100:00100000:3.0:1789760710.100897:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688505182606 time 688536890336 state 0x0 ref 1 00000100:00100000:3.0:1789760710.100911:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688537150752 time 688536890336 state 0x1 ref 1 00000100:00100000:3.0:1789760710.100940:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688537150752 time 688566230093 state 0x0 ref 1 00000100:00100000:3.0:1789760710.100952:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515607040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.100996:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19546112, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.101004:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.101023:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.101028:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.101034:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.101046:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760710.123623:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19546112 -> 19550208 00010000:00010000:3.0:1789760710.123641:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.123655:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.123665:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.123671:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760710.123676:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.123686:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.123703:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.123727:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d4e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.123812:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515607040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22856us (23048us total) trans 4294972089 rc 0/0 00000100:00100000:3.0:1789760710.123822:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4801 00000100:00100000:3.0:1789760710.123832:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688537150752 time 688566230093 state 0x2 ref 0 00000001:00080000:2.0:1789760710.123950:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972089 is committed 00000100:00100000:2.0:1789760710.127450:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.127644:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607168 00000100:00100000:3.0:1789760710.127653:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607168 00000100:00100000:2.0:1789760710.127784:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688537150752 time 688566230093 state 0x0 ref 1 00000100:00100000:2.0:1789760710.127798:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688566490509 time 688566230093 state 0x1 ref 1 00000100:00100000:2.0:1789760710.127855:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688566490509 time 688593143292 state 0x0 ref 1 00000100:00100000:2.0:1789760710.127875:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.127917:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19550208, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.127926:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.127940:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760710.127945:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.127951:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.127965:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.155542:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972090 is committed 00002000:00010000:3.0:1789760710.155633:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19550208 -> 19554304 00010000:00010000:3.0:1789760710.155647:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.155660:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.155670:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.155677:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760710.155683:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.155693:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.155770:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.155780:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.155996:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28117us (28549us total) trans 4294972090 rc 0/0 00000100:00100000:2.0:1789760710.156007:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4802 00000100:00100000:2.0:1789760710.156016:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688566490509 time 688593143292 state 0x2 ref 0 00000100:00100000:0.0:1789760710.159203:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.159356:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607296 00000100:00100000:2.0:1789760710.159365:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607296 00000100:00100000:2.0:1789760710.159608:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688566490509 time 688593143292 state 0x0 ref 1 00000100:00100000:2.0:1789760710.159625:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688593403708 time 688593143292 state 0x1 ref 1 00000100:00100000:2.0:1789760710.159682:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688593403708 time 688624971326 state 0x0 ref 1 00000100:00100000:2.0:1789760710.159696:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515607296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.159835:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19554304, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.159844:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.159859:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760710.159863:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.159869:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.159882:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760710.181749:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19554304 -> 19558400 00010000:00010000:2.0:1789760710.181767:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.181781:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.181790:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.181797:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760710.181802:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.181812:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.181829:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.181837:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d4f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.181912:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515607296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22214us (22762us total) trans 4294972091 rc 0/0 00000100:00100000:2.0:1789760710.181925:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4803 00000100:00100000:2.0:1789760710.181935:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688593403708 time 688624971326 state 0x2 ref 0 00000001:00080000:3.0:1789760710.181999:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972091 is committed 00000100:00100000:0.0:1789760710.185619:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.185690:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607424 00000100:00100000:2.0:1789760710.185699:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607424 00000100:00100000:2.0:1789760710.185763:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688593403708 time 688624971326 state 0x0 ref 1 00000100:00100000:2.0:1789760710.185777:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688625231742 time 688624971326 state 0x1 ref 1 00000100:00100000:2.0:1789760710.185803:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688625231742 time 688651092638 state 0x0 ref 1 00000100:00100000:2.0:1789760710.185814:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515607424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.185856:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19558400, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.185874:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.185889:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760710.185892:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.185898:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.185912:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.217203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972092 is committed 00002000:00010000:0.0:1789760710.217350:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19558400 -> 19562496 00010000:00010000:0.0:1789760710.217365:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.217379:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.217393:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.217401:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.217406:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.217416:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.217437:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.217445:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d4fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.217571:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515607424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31753us (31954us total) trans 4294972092 rc 0/0 00000100:00100000:0.0:1789760710.217581:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4804 00000100:00100000:0.0:1789760710.217589:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688625231742 time 688651092638 state 0x2 ref 0 00000100:00100000:0.0:1789760710.220692:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.220821:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607552 00000100:00100000:3.0:1789760710.220830:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607552 00000100:00100000:3.0:1789760710.220879:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688625231742 time 688651092638 state 0x0 ref 1 00000100:00100000:3.0:1789760710.220894:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688651353054 time 688651092638 state 0x1 ref 1 00000100:00100000:3.0:1789760710.220926:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688651353054 time 688686214756 state 0x0 ref 1 00000100:00100000:3.0:1789760710.220939:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.220984:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19562496, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.220993:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.221008:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.221012:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.221019:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.221038:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.248519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972093 is committed 00002000:00010000:0.0:1789760710.248650:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19562496 -> 19566592 00010000:00010000:0.0:1789760710.248666:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.248684:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.248693:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.248700:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.248721:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.248730:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.248750:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.248757:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d502 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.248844:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27901us (28156us total) trans 4294972093 rc 0/0 00000100:00100000:0.0:1789760710.248854:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4805 00000100:00100000:0.0:1789760710.248863:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688651353054 time 688686214756 state 0x2 ref 0 00000100:00100000:0.0:1789760710.256315:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.256418:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607680 00000100:00100000:3.0:1789760710.256428:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607680 00000100:00100000:3.0:1789760710.256478:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688651353054 time 688686214756 state 0x0 ref 1 00000100:00100000:3.0:1789760710.256492:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688686475172 time 688686214756 state 0x1 ref 1 00000100:00100000:3.0:1789760710.256559:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688686475172 time 688721846948 state 0x0 ref 1 00000100:00100000:3.0:1789760710.256571:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.256625:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19566592, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.256634:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.256650:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.256654:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.256661:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.256674:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.343968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972094 is committed 00002000:00010000:0.0:1789760710.344122:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19566592 -> 19570688 00010000:00010000:0.0:1789760710.344252:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.344266:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.344276:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.344283:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.344288:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.344297:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.344316:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.344325:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d509 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.344456:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 87882us (88145us total) trans 4294972094 rc 0/0 00000100:00100000:0.0:1789760710.344467:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4806 00000100:00100000:0.0:1789760710.344477:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688686475172 time 688721846948 state 0x2 ref 0 00000100:00100000:1.0:1789760710.348257:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.348351:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607808 00000100:00100000:3.0:1789760710.348361:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607808 00000100:00100000:3.0:1789760710.348411:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688686475172 time 688721846948 state 0x0 ref 1 00000100:00100000:3.0:1789760710.348424:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688722107364 time 688721846948 state 0x1 ref 1 00000100:00100000:3.0:1789760710.348477:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688722107364 time 688813765679 state 0x0 ref 1 00000100:00100000:3.0:1789760710.348489:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515607808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.348533:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19570688, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.348541:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.348556:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.348561:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.348567:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.348580:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760710.376621:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19570688 -> 19574784 00010000:00010000:3.0:1789760710.376642:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.376656:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.376672:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.376679:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760710.376684:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.376693:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.376728:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.376737:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d510 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19570688->19574783] (req 19570688->19574783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.376823:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515607808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28332us (28571us total) trans 4294972095 rc 0/0 00000100:00100000:3.0:1789760710.376833:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4807 00000100:00100000:3.0:1789760710.376843:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688722107364 time 688813765679 state 0x2 ref 0 00000001:00080000:0.0:1789760710.377896:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972095 is committed 00000100:00100000:2.0:1789760710.380565:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760710.381569:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515607936 00000100:00100000:0.0:1789760710.381581:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515607936 00000100:00100000:0.0:1789760710.381638:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688722107364 time 688813765679 state 0x0 ref 1 00000100:00100000:0.0:1789760710.381653:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688814026095 time 688813765679 state 0x1 ref 1 00000100:00100000:0.0:1789760710.381689:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688814026095 time 688846977897 state 0x0 ref 1 00000100:00100000:0.0:1789760710.381703:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760710.381787:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19574784, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:0.0:1789760710.381796:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.381813:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760710.381818:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.381825:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.381839:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.427522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972096 is committed 00002000:00010000:0.0:1789760710.427575:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19574784 -> 19578880 00010000:00010000:0.0:1789760710.427592:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.427611:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.427621:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.427630:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.427635:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.427644:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.427665:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.427673:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d517 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.427842:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515607936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46136us (47280us total) trans 4294972096 rc 0/0 00000100:00100000:0.0:1789760710.427853:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4808 00000100:00100000:0.0:1789760710.427864:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688814026095 time 688846977897 state 0x2 ref 0 00000100:00100000:2.0:1789760710.431577:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760710.431687:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608064 00000100:00100000:1.0:1789760710.431698:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608064 00000100:00100000:2.0:1789760710.431957:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688814026095 time 688846977897 state 0x0 ref 1 00000100:00100000:2.0:1789760710.431972:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688847238313 time 688846977897 state 0x1 ref 1 00000100:00100000:2.0:1789760710.432026:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688847238313 time 688897314938 state 0x0 ref 1 00000100:00100000:2.0:1789760710.432041:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515608064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.432086:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19578880, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.432095:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.432112:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760710.432117:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.432123:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.432198:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.454482:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972097 is committed 00002000:00010000:2.0:1789760710.454576:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19578880 -> 19582976 00010000:00010000:2.0:1789760710.454590:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.454603:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.454613:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.454621:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760710.454625:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.454635:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.454653:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.454662:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d51e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.454776:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515608064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22734us (23202us total) trans 4294972097 rc 0/0 00000100:00100000:2.0:1789760710.454785:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4809 00000100:00100000:2.0:1789760710.454794:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688847238313 time 688897314938 state 0x2 ref 0 00000100:00100000:0.0:1789760710.458682:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.458795:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608192 00000100:00100000:2.0:1789760710.458806:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608192 00000100:00100000:2.0:1789760710.458869:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688847238313 time 688897314938 state 0x0 ref 1 00000100:00100000:2.0:1789760710.458884:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688897575354 time 688897314938 state 0x1 ref 1 00000100:00100000:2.0:1789760710.458909:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688897575354 time 688924198657 state 0x0 ref 1 00000100:00100000:2.0:1789760710.458924:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515608192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.458970:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19582976, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.458979:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.458994:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760710.458999:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.459005:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.459019:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.491894:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972098 is committed 00002000:00010000:0.0:1789760710.492041:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19582976 -> 19587072 00010000:00010000:0.0:1789760710.492056:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492069:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492078:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492085:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.492090:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492100:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492118:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492127:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d525 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.492236:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515608192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33309us (33558us total) trans 4294972098 rc 0/0 00000100:00100000:0.0:1789760710.492246:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4810 00000100:00100000:0.0:1789760710.492833:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688897575354 time 688924198657 state 0x2 ref 0 00000100:00100000:0.0:1789760710.495937:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.496049:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608320 00000100:00100000:3.0:1789760710.496060:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608320 00000100:00100000:3.0:1789760710.496105:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688897575354 time 688924198657 state 0x0 ref 1 00000100:00100000:3.0:1789760710.496121:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688924459073 time 688924198657 state 0x1 ref 1 00000100:00100000:3.0:1789760710.496154:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688924459073 time 688961443716 state 0x0 ref 1 00000100:00100000:3.0:1789760710.496165:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515608320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.496210:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19587072, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.496228:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.496244:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760710.496257:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.496264:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.496278:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.521731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972099 is committed 00002000:00010000:3.0:1789760710.521827:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19587072 -> 19591168 00010000:00010000:3.0:1789760710.521842:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.521866:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.521877:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.521884:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760710.521890:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.521899:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.521919:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.521927:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d52c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.522011:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515608320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25844us (26079us total) trans 4294972099 rc 0/0 00000100:00100000:3.0:1789760710.522023:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4811 00000100:00100000:3.0:1789760710.522032:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688924459073 time 688961443716 state 0x2 ref 0 00000100:00100000:3.0:1789760710.526462:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760710.526606:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608448 00000100:00100000:0.0:1789760710.526616:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608448 00000100:00100000:0.0:1789760710.526668:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688924459073 time 688961443716 state 0x0 ref 1 00000100:00100000:0.0:1789760710.526682:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688961704132 time 688961443716 state 0x1 ref 1 00000100:00100000:0.0:1789760710.526743:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688961704132 time 688991995210 state 0x0 ref 1 00000100:00100000:0.0:1789760710.526757:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515608448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760710.526803:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19591168, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:0.0:1789760710.526812:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.526829:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760710.526833:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.526839:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.526853:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.557744:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972100 is committed 00002000:00010000:3.0:1789760710.557879:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19591168 -> 19595264 00010000:00010000:3.0:1789760710.557895:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.557908:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.557917:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.557931:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760710.557936:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.557946:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.557966:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.557974:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d533 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.558095:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515608448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31336us (31638us total) trans 4294972100 rc 0/0 00000100:00100000:3.0:1789760710.558105:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4812 00000100:00100000:3.0:1789760710.558115:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688961704132 time 688991995210 state 0x2 ref 0 00000100:00100000:1.0:1789760710.561115:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760710.561316:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608576 00000100:00100000:0.0:1789760710.561327:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608576 00000100:00100000:0.0:1789760710.561377:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688961704132 time 688991995210 state 0x0 ref 1 00000100:00100000:0.0:1789760710.561395:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688992255626 time 688991995210 state 0x1 ref 1 00000100:00100000:0.0:1789760710.561424:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688992255626 time 689026713390 state 0x0 ref 1 00000100:00100000:0.0:1789760710.561435:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cacc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515608576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760710.561478:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19595264, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:0.0:1789760710.561487:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.561502:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.561507:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.561513:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.561526:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.586088:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972101 is committed 00002000:00010000:1.0:1789760710.586187:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19595264 -> 19599360 00010000:00010000:1.0:1789760710.586203:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.586227:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.586238:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.586245:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760710.586249:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.586259:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.586282:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.586291:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d53a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.586381:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cacc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515608576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24942us (25270us total) trans 4294972101 rc 0/0 00000100:00100000:1.0:1789760710.586390:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4813 00000100:00100000:1.0:1789760710.586400:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688992255626 time 689026713390 state 0x2 ref 0 00000100:00100000:0.0:1789760710.590112:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.590208:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608704 00000100:00100000:2.0:1789760710.590232:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608704 00000100:00100000:2.0:1789760710.590288:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 688992255626 time 689026713390 state 0x0 ref 1 00000100:00100000:2.0:1789760710.590303:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689026973806 time 689026713390 state 0x1 ref 1 00000100:00100000:2.0:1789760710.590329:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689026973806 time 689055617789 state 0x0 ref 1 00000100:00100000:2.0:1789760710.590346:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515608704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.590392:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19599360, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.590401:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.590422:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.590427:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.590433:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.590448:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.614350:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972102 is committed 00002000:00010000:3.0:1789760710.614439:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19599360 -> 19603456 00010000:00010000:3.0:1789760710.614452:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.614462:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.614472:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.614477:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760710.614480:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.614486:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.614501:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.614506:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d541 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.614583:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515608704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24236us (24477us total) trans 4294972102 rc 0/0 00000100:00100000:3.0:1789760710.614590:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4814 00000100:00100000:3.0:1789760710.614597:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689026973806 time 689055617789 state 0x2 ref 0 00000100:00100000:0.0:1789760710.617279:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.617428:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608832 00000100:00100000:2.0:1789760710.617437:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608832 00000100:00100000:2.0:1789760710.617481:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689026973806 time 689055617789 state 0x0 ref 1 00000100:00100000:2.0:1789760710.617491:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689055878205 time 689055617789 state 0x1 ref 1 00000100:00100000:2.0:1789760710.617513:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689055878205 time 689082802423 state 0x0 ref 1 00000100:00100000:2.0:1789760710.617525:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515608832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.617568:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19603456, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.617575:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.617588:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.617591:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.617595:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.617608:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760710.638904:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19603456 -> 19607552 00010000:00010000:0.0:1789760710.638925:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638940:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638955:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638967:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.638972:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638982:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.639001:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.639009:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1d548 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19603456->19607551] (req 19603456->19607551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.639227:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515608832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21688us (21942us total) trans 4294972103 rc 0/0 00000100:00100000:0.0:1789760710.639239:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4815 00000100:00100000:0.0:1789760710.639249:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689055878205 time 689082802423 state 0x2 ref 0 00000001:00080000:2.0:1789760710.639390:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972103 is committed 00000100:00100000:1.0:1789760710.643226:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760710.644514:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515608960 00000100:00100000:0.0:1789760710.644527:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515608960 00000100:00100000:0.0:1789760710.644585:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689055878205 time 689082802423 state 0x0 ref 1 00000100:00100000:0.0:1789760710.644597:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689083062839 time 689082802423 state 0x1 ref 1 00000100:00100000:0.0:1789760710.644631:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689083062839 time 689109919658 state 0x0 ref 1 00000100:00100000:0.0:1789760710.644644:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515608960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760710.644697:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19607552, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:0.0:1789760710.644705:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.644732:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.644735:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.644740:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.644750:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.672039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972104 is committed 00002000:00010000:1.0:1789760710.672364:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19607552 -> 19611648 00010000:00010000:1.0:1789760710.672381:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.672394:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.672404:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.672411:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760710.672416:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.672427:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.672446:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.672453:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d54f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.672587:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515608960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27941us (29378us total) trans 4294972104 rc 0/0 00000100:00100000:1.0:1789760710.672598:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4816 00000100:00100000:1.0:1789760710.672608:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689083062839 time 689109919658 state 0x2 ref 0 00000100:00100000:1.0:1789760710.678058:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.678206:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609088 00000100:00100000:3.0:1789760710.678227:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609088 00000100:00100000:3.0:1789760710.678288:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689083062839 time 689109919658 state 0x0 ref 1 00000100:00100000:3.0:1789760710.678304:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689110180074 time 689109919658 state 0x1 ref 1 00000100:00100000:3.0:1789760710.678341:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689110180074 time 689143630446 state 0x0 ref 1 00000100:00100000:3.0:1789760710.678355:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.678404:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19611648, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.678415:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.678432:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760710.678437:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.678445:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.678460:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.699518:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972105 is committed 00002000:00010000:1.0:1789760710.699657:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19611648 -> 19615744 00010000:00010000:1.0:1789760710.699672:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.699686:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.699696:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.699725:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760710.699730:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.699739:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.699757:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.699764:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d556 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.699888:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21532us (21835us total) trans 4294972105 rc 0/0 00000100:00100000:1.0:1789760710.699899:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4817 00000100:00100000:1.0:1789760710.699909:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689110180074 time 689143630446 state 0x2 ref 0 00000100:00100000:2.0:1789760710.709124:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.709422:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609216 00000100:00100000:3.0:1789760710.709431:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609216 00000100:00100000:3.0:1789760710.709475:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689110180074 time 689143630446 state 0x0 ref 1 00000100:00100000:3.0:1789760710.709487:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689143890862 time 689143630446 state 0x1 ref 1 00000100:00100000:0.0:1789760710.709604:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689143890862 time 689174891966 state 0x0 ref 1 00000100:00100000:0.0:1789760710.709615:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760710.709653:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19615744, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:0.0:1789760710.709660:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.709672:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.709675:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.709679:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.709689:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.743874:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972106 is committed 00002000:00010000:0.0:1789760710.743964:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19615744 -> 19619840 00010000:00010000:0.0:1789760710.743980:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.743992:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.744001:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.744008:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.744013:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.744022:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.744041:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.744049:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d55d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.744156:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34538us (35038us total) trans 4294972106 rc 0/0 00000100:00100000:0.0:1789760710.744167:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4818 00000100:00100000:0.0:1789760710.744177:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689143890862 time 689174891966 state 0x2 ref 0 00000100:00100000:0.0:1789760710.751088:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.751230:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609344 00000100:00100000:2.0:1789760710.751241:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609344 00000100:00100000:2.0:1789760710.751293:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689143890862 time 689174891966 state 0x0 ref 1 00000100:00100000:2.0:1789760710.751308:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689175152382 time 689174891966 state 0x1 ref 1 00000100:00100000:2.0:1789760710.751337:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689175152382 time 689216626762 state 0x0 ref 1 00000100:00100000:2.0:1789760710.751350:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.751404:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19619840, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.751413:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.751428:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.751432:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.751446:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.751460:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.789533:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972107 is committed 00002000:00010000:1.0:1789760710.789589:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19619840 -> 19623936 00010000:00010000:1.0:1789760710.789606:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.789620:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.789636:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.789643:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760710.789648:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.789658:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.789676:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.789685:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d564 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.790538:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39183us (39450us total) trans 4294972107 rc 0/0 00000100:00100000:1.0:1789760710.790940:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4819 00000100:00100000:1.0:1789760710.790950:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689175152382 time 689216626762 state 0x2 ref 0 00000100:00100000:1.0:1789760710.792924:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760710.792979:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609472 00000100:00100000:2.0:1789760710.792989:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609472 00000100:00100000:2.0:1789760710.793045:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689175152382 time 689216626762 state 0x0 ref 1 00000100:00100000:2.0:1789760710.793060:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689216887178 time 689216626762 state 0x1 ref 1 00000100:00100000:2.0:1789760710.793085:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689216887178 time 689258373639 state 0x0 ref 1 00000100:00100000:2.0:1789760710.793097:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760710.793141:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19623936, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:2.0:1789760710.793148:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.793160:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.793163:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.793168:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.793178:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760710.812123:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19623936 -> 19628032 00010000:00010000:0.0:1789760710.812153:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.812166:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.812176:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.812184:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760710.812188:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.812198:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.812226:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.812235:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d56b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.812332:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515609472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 19232us (19407us total) trans 4294972108 rc 0/0 00000100:00100000:0.0:1789760710.812342:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4820 00000100:00100000:0.0:1789760710.812352:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689216887178 time 689258373639 state 0x2 ref 0 00000001:00080000:1.0:1789760710.812862:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972108 is committed 00000100:00100000:2.0:1789760710.817942:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760710.818062:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609600 00000100:00100000:3.0:1789760710.818073:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609600 00000100:00100000:3.0:1789760710.818137:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689216887178 time 689258373639 state 0x0 ref 1 00000100:00100000:3.0:1789760710.818151:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689258634055 time 689258373639 state 0x1 ref 1 00000100:00100000:3.0:1789760710.818233:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689258634055 time 689283522180 state 0x0 ref 1 00000100:00100000:3.0:1789760710.818247:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515609600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760710.818298:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19628032, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9807 00010000:00010000:3.0:1789760710.818307:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.818322:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760710.818326:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.818331:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.818345:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.883027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972109 is committed 00002000:00010000:2.0:1789760710.883089:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19628032 -> 19632128 00002000:00010000:2.0:1789760710.883093:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760710 -> 1789760711 00002000:00010000:2.0:1789760710.883096:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760710 -> 1789760711 00010000:00010000:2.0:1789760710.883109:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.883123:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.883253:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.883329:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760710.883335:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.883346:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.883365:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.883373:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d572 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.883472:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515609600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65223us (65538us total) trans 4294972109 rc 0/0 00000100:00100000:2.0:1789760710.883483:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4821 00000100:00100000:2.0:1789760710.883492:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689258634055 time 689283522180 state 0x2 ref 0 00000100:00100000:2.0:1789760710.889691:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760710.889805:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609728 00000100:00100000:1.0:1789760710.889815:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609728 00000100:00100000:1.0:1789760710.890226:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689258634055 time 689283522180 state 0x0 ref 1 00000100:00100000:1.0:1789760710.890242:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689283782596 time 689283522180 state 0x1 ref 1 00000100:00100000:1.0:1789760710.890379:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689283782596 time 689355666192 state 0x0 ref 1 00000100:00100000:1.0:1789760710.890393:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760710.890441:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19632128, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:1.0:1789760710.890449:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.890465:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760710.890470:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.890476:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.890491:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.933161:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972110 is committed 00002000:00010000:1.0:1789760710.933435:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19632128 -> 19636224 00010000:00010000:1.0:1789760710.933450:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.933464:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.933473:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.933481:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760710.933486:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.933495:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.933515:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.933522:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d579 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.933618:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515609728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43222us (43929us total) trans 4294972110 rc 0/0 00000100:00100000:1.0:1789760710.933628:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4822 00000100:00100000:1.0:1789760710.933638:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689283782596 time 689355666192 state 0x2 ref 0 00000100:00100000:3.0:1789760710.942962:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760710.943151:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609856 00000100:00100000:1.0:1789760710.943161:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609856 00000100:00100000:1.0:1789760710.944102:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689283782596 time 689355666192 state 0x0 ref 1 00000100:00100000:1.0:1789760710.944123:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689355926608 time 689355666192 state 0x1 ref 1 00000100:00100000:1.0:1789760710.944205:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689355926608 time 689409493112 state 0x0 ref 1 00000100:00100000:1.0:1789760710.944233:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515609856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760710.944392:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19636224, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:1.0:1789760710.944401:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.944417:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760710.944421:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.944427:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.944442:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.974279:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760711.007913:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972111 is committed 00002000:00010000:1.0:1789760711.008272:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19636224 -> 19640320 00010000:00010000:1.0:1789760711.008288:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.008302:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.008311:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.008318:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.008322:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.008332:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.008350:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.008357:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d580 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.008447:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515609856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64215us (65490us total) trans 4294972111 rc 0/0 00000100:00100000:1.0:1789760711.008457:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4823 00000100:00100000:1.0:1789760711.008466:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689355926608 time 689409493112 state 0x2 ref 0 00000100:00100000:3.0:1789760711.017898:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.017999:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515609984 00000100:00100000:0.0:1789760711.018008:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515609984 00000100:00100000:0.0:1789760711.018060:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689355926608 time 689409493112 state 0x0 ref 1 00000100:00100000:0.0:1789760711.018075:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689409753528 time 689409493112 state 0x1 ref 1 00000100:00100000:0.0:1789760711.018107:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689409753528 time 689483395766 state 0x0 ref 1 00000100:00100000:0.0:1789760711.018118:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515609984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.018172:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19640320, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.018181:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.018197:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760711.018201:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.018207:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.018232:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.058923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972112 is committed 00002000:00010000:0.0:1789760711.059074:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19640320 -> 19644416 00010000:00010000:0.0:1789760711.059089:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.059100:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.059109:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.059116:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760711.059121:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.059140:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.059161:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.059169:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d587 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.059688:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515609984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41565us (41792us total) trans 4294972112 rc 0/0 00000100:00100000:0.0:1789760711.059921:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4824 00000100:00100000:0.0:1789760711.059931:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689409753528 time 689483395766 state 0x2 ref 0 00000100:00100000:2.0:1789760711.062382:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.062462:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610112 00000100:00100000:1.0:1789760711.062472:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610112 00000100:00100000:1.0:1789760711.062519:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689409753528 time 689483395766 state 0x0 ref 1 00000100:00100000:1.0:1789760711.062534:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689483656182 time 689483395766 state 0x1 ref 1 00000100:00100000:1.0:1789760711.062563:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689483656182 time 689527852304 state 0x0 ref 1 00000100:00100000:1.0:1789760711.062574:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515610112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760711.062619:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19644416, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:1.0:1789760711.062628:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.062644:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760711.062648:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.062654:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.062667:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.096058:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19644416 -> 19648512 00010000:00010000:1.0:1789760711.096474:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.096487:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.096496:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.096503:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.096507:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.096517:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.096540:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.096548:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d58e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.096645:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515610112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34067us (34265us total) trans 4294972113 rc 0/0 00000100:00100000:1.0:1789760711.096657:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4825 00000100:00100000:1.0:1789760711.096667:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689483656182 time 689527852304 state 0x2 ref 0 00000001:00080000:1.0:1789760711.096744:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972113 is committed 00000100:00100000:3.0:1789760711.104521:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.104986:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610240 00000100:00100000:0.0:1789760711.104995:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610240 00000100:00100000:0.0:1789760711.105048:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689483656182 time 689527852304 state 0x0 ref 1 00000100:00100000:0.0:1789760711.105062:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689528112720 time 689527852304 state 0x1 ref 1 00000100:00100000:0.0:1789760711.105092:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689528112720 time 689570381684 state 0x0 ref 1 00000100:00100000:0.0:1789760711.105104:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515610240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.105150:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19648512, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.105159:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.105174:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760711.105178:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.105184:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.105198:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760711.155330:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19648512 -> 19652608 00010000:00010000:0.0:1789760711.155349:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.155363:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.155373:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.155380:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760711.155385:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.155393:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.155532:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.155541:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d595 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.155629:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515610240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50522us (51114us total) trans 4294972114 rc 0/0 00000100:00100000:0.0:1789760711.155643:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4826 00000100:00100000:0.0:1789760711.155653:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689528112720 time 689570381684 state 0x2 ref 0 00000001:00080000:0.0:1789760711.156676:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972114 is committed 00000100:00100000:2.0:1789760711.163017:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.163155:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610368 00000100:00100000:1.0:1789760711.163164:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610368 00000100:00100000:1.0:1789760711.163472:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689528112720 time 689570381684 state 0x0 ref 1 00000100:00100000:1.0:1789760711.163490:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689570642100 time 689570381684 state 0x1 ref 1 00000100:00100000:1.0:1789760711.163556:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689570642100 time 689628844510 state 0x0 ref 1 00000100:00100000:1.0:1789760711.163570:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515610368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760711.163618:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19652608, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:1.0:1789760711.163627:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.163648:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760711.163653:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.163660:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.163673:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.166206:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544405760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.166290:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760711.167459:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544405760 00000100:00100000:3.0:1789760711.167470:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544405760 00000100:00100000:3.0:1789760711.167637:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:3.0:1789760711.167648:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9092de450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544405760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.167803:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9092de450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544405760:12345-0@lo:400:kworker.0 Request processed in 157us (1511us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760711.167811:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:0.0:1789760711.168521:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544405888:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.168565:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760711.168591:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544406400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.168611:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760711.168631:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760711.168636:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544406016:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.168656:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760711.168691:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544405888 00000100:00100000:0.0:1789760711.168695:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544405888 00000100:00100000:0.0:1789760711.168823:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406016 00000100:00100000:0.0:1789760711.168827:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406016 00000100:00100000:0.0:1789760711.168856:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1789760711.168864:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544405888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.168907:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544405888:12345-0@lo:400:kworker.0 Request processed in 44us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760711.168913:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1789760711.168927:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789760711.168932:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544406016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.168955:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544406016:12345-0@lo:400:kworker.0 Request processed in 23us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760711.168961:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789760711.168977:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406400 00000100:00100000:0.0:1789760711.168980:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406400 00000100:00100000:0.0:1789760711.169005:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:0.0:1789760711.169010:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544406400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.169042:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544406400:12345-0@lo:400:kworker.0 Request processed in 31us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760711.169048:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00080000:0.0:1789760711.169068:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760711.169073:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760711.169100:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544406016:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.169110:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544405888:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.169118:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544406400:0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.174924:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544406144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.174992:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760711.175036:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760711.175043:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544406272:0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.175067:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760711.175100:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406144 00000100:00100000:3.0:1789760711.175106:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406144 00000100:00100000:3.0:1789760711.175388:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00080000:0.0:1789760711.175586:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760711.175622:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544406144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.175735:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544406144:12345-0@lo:400:kworker.0 Request processed in 339us (743us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760711.175744:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:3.0:1789760711.175767:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406272 00000100:00100000:3.0:1789760711.175771:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406272 00000100:00100000:3.0:1789760711.175803:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789760711.175809:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544406272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760711.175839:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544406272:12345-0@lo:400:kworker.0 Request processed in 30us (772us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760711.175845:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00080000:3.0:1789760711.175863:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760711.175886:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544405760:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.177828:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544406144:0@lo:400:kworker.0 00000100:00100000:0.0:1789760711.177857:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544406272:0@lo:400:kworker.0 00002000:00010000:2.0:1789760711.208836:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19652608 -> 19656704 00010000:00010000:2.0:1789760711.208856:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.208869:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.208879:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.208886:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760711.208891:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.208900:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.208919:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.208926:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d59c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.209022:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515610368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45450us (46008us total) trans 4294972115 rc 0/0 00000100:00100000:2.0:1789760711.209033:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4827 00000100:00100000:2.0:1789760711.209043:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689570642100 time 689628844510 state 0x2 ref 0 00000001:00080000:3.0:1789760711.209699:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972115 is committed 00000100:00100000:2.0:1789760711.217490:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.217703:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610496 00000100:00100000:1.0:1789760711.217730:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610496 00000100:00100000:1.0:1789760711.217784:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689570642100 time 689628844510 state 0x0 ref 1 00000100:00100000:1.0:1789760711.217799:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689629104926 time 689628844510 state 0x1 ref 1 00000100:00100000:1.0:1789760711.217828:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689629104926 time 689683117169 state 0x0 ref 1 00000100:00100000:1.0:1789760711.217841:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515610496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760711.217898:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19656704, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:1.0:1789760711.217907:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217923:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760711.217927:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.217933:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.217953:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.261106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972116 is committed 00002000:00010000:3.0:1789760711.261182:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19656704 -> 19660800 00010000:00010000:3.0:1789760711.261198:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.261222:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.261232:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.261240:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760711.261245:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.261254:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.261275:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.261283:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d5a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.261389:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515610496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43544us (43903us total) trans 4294972116 rc 0/0 00000100:00100000:3.0:1789760711.261400:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4828 00000100:00100000:3.0:1789760711.261411:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689629104926 time 689683117169 state 0x2 ref 0 00000100:00100000:1.0:1789760711.267065:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.267169:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610624 00000100:00100000:0.0:1789760711.267179:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610624 00000100:00100000:0.0:1789760711.267242:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689629104926 time 689683117169 state 0x0 ref 1 00000100:00100000:0.0:1789760711.267257:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689683377585 time 689683117169 state 0x1 ref 1 00000100:00100000:0.0:1789760711.267288:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689683377585 time 689732577329 state 0x0 ref 1 00000100:00100000:0.0:1789760711.267301:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515610624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.267359:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19660800, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.267368:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.267384:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760711.267388:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.267394:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.267409:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.306406:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972117 is committed 00002000:00010000:1.0:1789760711.306503:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19660800 -> 19664896 00010000:00010000:1.0:1789760711.306522:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.306537:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.306547:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.306555:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.306559:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.306568:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.306590:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.306598:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d5aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.307058:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515610624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39753us (39996us total) trans 4294972117 rc 0/0 00000100:00100000:1.0:1789760711.307071:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4829 00000100:00100000:1.0:1789760711.307082:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689683377585 time 689732577329 state 0x2 ref 0 00000100:00100000:2.0:1789760711.310549:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.310802:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610752 00000100:00100000:0.0:1789760711.310812:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610752 00000100:00100000:0.0:1789760711.310952:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689683377585 time 689732577329 state 0x0 ref 1 00000100:00100000:0.0:1789760711.310968:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689732837745 time 689732577329 state 0x1 ref 1 00000100:00100000:0.0:1789760711.310997:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689732837745 time 689776286108 state 0x0 ref 1 00000100:00100000:0.0:1789760711.311009:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515610752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.311060:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19664896, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.311068:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.311083:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760711.311088:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.311094:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.311108:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.349661:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19664896 -> 19668992 00010000:00010000:1.0:1789760711.349682:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.349684:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972118 is committed 00010000:00010000:1.0:1789760711.349697:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.349746:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.349755:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.349759:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.350106:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.350128:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.350141:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d5b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.350240:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515610752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39228us (39695us total) trans 4294972118 rc 0/0 00000100:00100000:1.0:1789760711.350250:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4830 00000100:00100000:1.0:1789760711.350260:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689732837745 time 689776286108 state 0x2 ref 0 00000100:00100000:0.0:1789760711.358589:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760711.359840:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515610880 00000100:00100000:2.0:1789760711.359854:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515610880 00000100:00100000:2.0:1789760711.359914:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689732837745 time 689776286108 state 0x0 ref 1 00000100:00100000:2.0:1789760711.359930:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689776546524 time 689776286108 state 0x1 ref 1 00000100:00100000:2.0:1789760711.359962:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689776546524 time 689825251271 state 0x0 ref 1 00000100:00100000:2.0:1789760711.359974:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515610880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760711.360023:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19668992, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:2.0:1789760711.360032:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.360047:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760711.360052:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.360059:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.360079:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760711.385608:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19668992 -> 19673088 00010000:00010000:3.0:1789760711.385626:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.385639:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.385649:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.385656:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760711.385661:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.385671:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.385688:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.385697:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d5b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.385874:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515610880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25898us (27289us total) trans 4294972119 rc 0/0 00000100:00100000:3.0:1789760711.385885:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4831 00000100:00100000:3.0:1789760711.385895:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689776546524 time 689825251271 state 0x2 ref 0 00000001:00080000:3.0:1789760711.385932:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972119 is committed 00000100:00100000:3.0:1789760711.389362:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760711.389596:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611008 00000100:00100000:2.0:1789760711.389606:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611008 00000100:00100000:2.0:1789760711.389654:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689776546524 time 689825251271 state 0x0 ref 1 00000100:00100000:2.0:1789760711.389668:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689825511687 time 689825251271 state 0x1 ref 1 00000100:00100000:2.0:1789760711.389697:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689825511687 time 689854986673 state 0x0 ref 1 00000100:00100000:2.0:1789760711.389859:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515611008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760711.389905:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19673088, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:2.0:1789760711.389913:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.389929:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760711.389934:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.389940:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.389954:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.417160:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972120 is committed 00002000:00010000:0.0:1789760711.417461:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19673088 -> 19677184 00010000:00010000:0.0:1789760711.417477:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.417489:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.417497:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.417504:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760711.417509:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.417518:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.417537:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.417544:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d5bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.417641:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515611008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27929us (28283us total) trans 4294972120 rc 0/0 00000100:00100000:0.0:1789760711.417798:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4832 00000100:00100000:0.0:1789760711.417808:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689825511687 time 689854986673 state 0x2 ref 0 00000100:00100000:2.0:1789760711.425205:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.425339:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611136 00000100:00100000:1.0:1789760711.425349:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611136 00000100:00100000:1.0:1789760711.425403:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689825511687 time 689854986673 state 0x0 ref 1 00000100:00100000:1.0:1789760711.425418:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689855247089 time 689854986673 state 0x1 ref 1 00000100:00100000:1.0:1789760711.425449:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689855247089 time 689890738935 state 0x0 ref 1 00000100:00100000:1.0:1789760711.425462:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515611136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760711.425508:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19677184, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:1.0:1789760711.425517:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.425533:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:1.0:1789760711.425537:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.425543:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.425557:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.478109:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972121 is committed 00002000:00010000:1.0:1789760711.478238:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19677184 -> 19681280 00010000:00010000:1.0:1789760711.478252:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.478265:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.478273:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.478280:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.478285:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.478294:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.478318:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.478326:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d5c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.478451:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515611136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52986us (53250us total) trans 4294972121 rc 0/0 00000100:00100000:1.0:1789760711.478462:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4833 00000100:00100000:1.0:1789760711.478471:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689855247089 time 689890738935 state 0x2 ref 0 00000100:00100000:1.0:1789760711.482517:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760711.482597:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611264 00000100:00100000:3.0:1789760711.482607:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611264 00000100:00100000:3.0:1789760711.482659:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689855247089 time 689890738935 state 0x0 ref 1 00000100:00100000:3.0:1789760711.482674:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689890999351 time 689890738935 state 0x1 ref 1 00000100:00100000:3.0:1789760711.482750:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689890999351 time 689948038462 state 0x0 ref 1 00000100:00100000:3.0:1789760711.482763:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515611264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760711.482811:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19681280, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:3.0:1789760711.482820:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.482837:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760711.482842:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.482848:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.482874:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.510259:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972122 is committed 00002000:00010000:1.0:1789760711.510521:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19681280 -> 19685376 00010000:00010000:1.0:1789760711.510537:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.510550:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.510559:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.510567:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.510572:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.510582:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.510747:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.510763:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d5cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.511267:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515611264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28500us (28754us total) trans 4294972122 rc 0/0 00000100:00100000:1.0:1789760711.511279:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4834 00000100:00100000:1.0:1789760711.511290:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689890999351 time 689948038462 state 0x2 ref 0 00000100:00100000:3.0:1789760711.517407:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.518051:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611392 00000100:00100000:0.0:1789760711.518061:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611392 00000100:00100000:0.0:1789760711.518110:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689890999351 time 689948038462 state 0x0 ref 1 00000100:00100000:0.0:1789760711.518136:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689948298878 time 689948038462 state 0x1 ref 1 00000100:00100000:0.0:1789760711.518166:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689948298878 time 689983455601 state 0x0 ref 1 00000100:00100000:0.0:1789760711.518178:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515611392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.518235:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19685376, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.518243:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.518259:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760711.518263:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.518270:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.518283:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.547301:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972123 is committed 00002000:00010000:0.0:1789760711.547912:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19685376 -> 19689472 00010000:00010000:0.0:1789760711.547929:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.547942:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.547952:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.547959:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760711.548013:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.548025:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.548048:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.548057:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d5d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.548253:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515611392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30070us (30849us total) trans 4294972123 rc 0/0 00000100:00100000:0.0:1789760711.548267:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4835 00000100:00100000:0.0:1789760711.548277:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689948298878 time 689983455601 state 0x2 ref 0 00000100:00100000:2.0:1789760711.552917:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.552992:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611520 00000100:00100000:0.0:1789760711.553002:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611520 00000100:00100000:0.0:1789760711.553059:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689948298878 time 689983455601 state 0x0 ref 1 00000100:00100000:0.0:1789760711.553074:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689983716017 time 689983455601 state 0x1 ref 1 00000100:00100000:0.0:1789760711.553094:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689983716017 time 690018384031 state 0x0 ref 1 00000100:00100000:0.0:1789760711.553106:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515611520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.553154:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19689472, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.553162:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.553177:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.553182:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.553187:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.553201:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.605681:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.605737:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.605807:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760711.605837:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611648 00000100:00100000:1.0:1789760711.605846:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611648 00000100:00100000:1.0:1789760711.605891:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 380 00000100:00100000:1.0:1789760711.605901:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9092dec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2363:x1876699515611648:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760711.605962:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9092dec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2363:x1876699515611648:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 62us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760711.605969:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 380 00000100:00100000:1.0:1789760711.606000:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611776 00000100:00100000:1.0:1789760711.606004:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611776 00000100:00100000:1.0:1789760711.606150:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515611904 00000100:00100000:1.0:1789760711.606155:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515611904 00000100:00100000:1.0:1789760711.606183:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 426 00000100:00100000:1.0:1789760711.606190:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515611776:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760711.606236:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515611776:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 46us (498us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760711.606252:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 426 00000100:00100000:1.0:1789760711.606264:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 427 00000100:00100000:1.0:1789760711.606271:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515611904:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760711.606294:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515611904:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 24us (487us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760711.606300:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 427 00000100:00080000:1.0:1789760711.606402:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:0.0:1789760711.609221:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19689472 -> 19693568 00010000:00010000:0.0:1789760711.609250:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.609264:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.609274:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.609283:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760711.609288:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.609297:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.609317:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.609326:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.609539:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515611520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56429us (56626us total) trans 4294972124 rc 0/0 00000100:00100000:0.0:1789760711.609549:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4836 00000100:00100000:0.0:1789760711.609558:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689983716017 time 690018384031 state 0x2 ref 0 00000001:00080000:1.0:1789760711.610183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972124 is committed 00000100:00100000:1.0:1789760711.614548:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760711.615220:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612032 00000100:00100000:2.0:1789760711.615230:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612032 00000100:00100000:2.0:1789760711.615290:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 689983716017 time 690018384031 state 0x0 ref 1 00000100:00100000:2.0:1789760711.615306:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690018644447 time 690018384031 state 0x1 ref 1 00000100:00100000:2.0:1789760711.615338:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690018644447 time 690080627524 state 0x0 ref 1 00000100:00100000:2.0:1789760711.615351:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515612032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760711.615399:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19693568, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:2.0:1789760711.615407:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.615422:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.615426:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.615432:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.615445:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.645585:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972125 is committed 00002000:00010000:2.0:1789760711.645894:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19693568 -> 19697664 00010000:00010000:2.0:1789760711.645910:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645930:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645939:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645946:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760711.645951:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645961:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645980:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.645989:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.646081:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515612032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30727us (31538us total) trans 4294972125 rc 0/0 00000100:00100000:2.0:1789760711.646099:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4837 00000100:00100000:2.0:1789760711.646109:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690018644447 time 690080627524 state 0x2 ref 0 00000100:00100000:2.0:1789760711.653278:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.654693:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612160 00000100:00100000:0.0:1789760711.654738:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612160 00000100:00100000:0.0:1789760711.654799:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690018644447 time 690080627524 state 0x0 ref 1 00000100:00100000:0.0:1789760711.654814:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690080887940 time 690080627524 state 0x1 ref 1 00000100:00100000:0.0:1789760711.654848:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690080887940 time 690120136043 state 0x0 ref 1 00000100:00100000:0.0:1789760711.654860:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515612160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.654908:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19697664, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:0.0:1789760711.654918:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.654932:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.654936:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.654942:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.654956:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.740418:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972126 is committed 00002000:00010000:2.0:1789760711.740559:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19697664 -> 19701760 00010000:00010000:2.0:1789760711.740575:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.740586:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.740594:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.740600:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760711.740604:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.740612:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.740630:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.740638:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d5e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.740761:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515612160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 85899us (87488us total) trans 4294972126 rc 0/0 00000100:00100000:2.0:1789760711.740772:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4838 00000100:00100000:2.0:1789760711.740781:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690080887940 time 690120136043 state 0x2 ref 0 00000100:00100000:1.0:1789760711.744965:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760711.745199:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612288 00000100:00100000:2.0:1789760711.745220:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612288 00000100:00100000:2.0:1789760711.745274:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690080887940 time 690120136043 state 0x0 ref 1 00000100:00100000:2.0:1789760711.745289:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690120396459 time 690120136043 state 0x1 ref 1 00000100:00100000:2.0:1789760711.745314:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690120396459 time 690210602773 state 0x0 ref 1 00000100:00100000:2.0:1789760711.745326:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515612288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760711.745375:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19701760, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:2.0:1789760711.745385:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.745400:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.745404:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.745410:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.745423:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760711.780771:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19701760 -> 19705856 00010000:00010000:1.0:1789760711.780791:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.780805:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.780815:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.780822:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760711.780827:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.780843:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.780863:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.780870:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1d5f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.780956:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515612288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35627us (35995us total) trans 4294972127 rc 0/0 00000100:00100000:1.0:1789760711.780966:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4839 00000100:00100000:1.0:1789760711.780976:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690120396459 time 690210602773 state 0x2 ref 0 00000001:00080000:2.0:1789760711.782218:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972127 is committed 00000100:00100000:1.0:1789760711.784359:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760711.784457:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612416 00000100:00100000:3.0:1789760711.784468:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612416 00000100:00100000:3.0:1789760711.784516:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690120396459 time 690210602773 state 0x0 ref 1 00000100:00100000:3.0:1789760711.784530:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690210863189 time 690210602773 state 0x1 ref 1 00000100:00100000:3.0:1789760711.784562:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690210863189 time 690249850475 state 0x0 ref 1 00000100:00100000:3.0:1789760711.784575:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515612416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760711.784620:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19705856, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:3.0:1789760711.784629:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.784645:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760711.784649:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.784657:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.784671:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760711.815125:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19705856 -> 19709952 00010000:00010000:0.0:1789760711.815150:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.815161:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.815174:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.815181:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760711.815185:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.815193:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.815220:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.815228:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d5f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.815354:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515612416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30776us (30998us total) trans 4294972128 rc 0/0 00000100:00100000:0.0:1789760711.815364:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4840 00000100:00100000:0.0:1789760711.815373:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690210863189 time 690249850475 state 0x2 ref 0 00000001:00080000:2.0:1789760711.815514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972128 is committed 00000100:00100000:3.0:1789760711.821196:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760711.823298:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612544 00000100:00100000:2.0:1789760711.823311:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612544 00000100:00100000:3.0:1789760711.827275:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690210863189 time 690249850475 state 0x0 ref 1 00000100:00100000:3.0:1789760711.827293:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690250110891 time 690249850475 state 0x1 ref 1 00000100:00100000:3.0:1789760711.827389:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690250110891 time 690292677033 state 0x0 ref 1 00000100:00100000:3.0:1789760711.827405:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515612544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760711.827459:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19709952, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9807 00010000:00010000:3.0:1789760711.827469:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.827484:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760711.827488:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.827494:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.827506:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.866638:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972129 is committed 00002000:00010000:3.0:1789760711.867296:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19709952 -> 19714048 00002000:00010000:3.0:1789760711.867302:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760711 -> 1789760712 00002000:00010000:3.0:1789760711.867304:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760711 -> 1789760712 00010000:00010000:3.0:1789760711.867317:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.867330:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.867339:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.867369:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760711.867374:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.867383:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.867748:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.867755:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d5fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.867847:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515612544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40442us (46656us total) trans 4294972129 rc 0/0 00000100:00100000:3.0:1789760711.867857:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4841 00000100:00100000:3.0:1789760711.867867:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690250110891 time 690292677033 state 0x2 ref 0 00000100:00100000:1.0:1789760711.878327:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760711.878506:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612672 00000100:00100000:0.0:1789760711.878515:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612672 00000100:00100000:0.0:1789760711.878570:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690250110891 time 690292677033 state 0x0 ref 1 00000100:00100000:0.0:1789760711.878584:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690292937449 time 690292677033 state 0x1 ref 1 00000100:00100000:0.0:1789760711.878608:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690292937449 time 690343898088 state 0x0 ref 1 00000100:00100000:0.0:1789760711.878619:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515612672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760711.878664:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19714048, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:0.0:1789760711.878673:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.878688:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.878692:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.878698:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.878813:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.930478:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972130 is committed 00002000:00010000:3.0:1789760711.930638:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19714048 -> 19718144 00010000:00010000:3.0:1789760711.931372:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931387:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931402:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931410:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760711.931415:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931425:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931447:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931455:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d605 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.931923:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515612672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53298us (53596us total) trans 4294972130 rc 0/0 00000100:00100000:2.0:1789760711.931946:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4842 00000100:00100000:2.0:1789760711.931956:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690292937449 time 690343898088 state 0x2 ref 0 00000100:00100000:1.0:1789760711.937902:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760711.938341:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612800 00000100:00100000:2.0:1789760711.938352:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612800 00000100:00100000:2.0:1789760711.938405:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690292937449 time 690343898088 state 0x0 ref 1 00000100:00100000:2.0:1789760711.938421:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690344158504 time 690343898088 state 0x1 ref 1 00000100:00100000:2.0:1789760711.938453:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690344158504 time 690403742488 state 0x0 ref 1 00000100:00100000:2.0:1789760711.938465:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515612800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760711.938511:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19718144, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:2.0:1789760711.938520:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.938537:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.938541:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.938547:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.938560:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.998945:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760712.013417:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19718144 -> 19722240 00010000:00010000:3.0:1789760712.013438:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.013451:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.013459:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.013471:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760712.013477:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.013486:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.013505:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.013512:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d60c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.013608:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515612800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75140us (75711us total) trans 4294972131 rc 0/0 00000100:00100000:3.0:1789760712.013618:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4843 00000100:00100000:3.0:1789760712.013626:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690344158504 time 690403742488 state 0x2 ref 0 00000001:00080000:1.0:1789760712.013957:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972131 is committed 00000100:00100000:3.0:1789760712.017283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760712.017486:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515612928 00000100:00100000:2.0:1789760712.017496:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515612928 00000100:00100000:2.0:1789760712.017567:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690344158504 time 690403742488 state 0x0 ref 1 00000100:00100000:2.0:1789760712.017583:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690404002904 time 690403742488 state 0x1 ref 1 00000100:00100000:2.0:1789760712.017610:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690404002904 time 690482899859 state 0x0 ref 1 00000100:00100000:2.0:1789760712.017622:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515612928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760712.017668:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19722240, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:2.0:1789760712.017676:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.017692:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760712.017697:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.017704:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.017755:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760712.018664:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760712.108425:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972132 is committed 00002000:00010000:2.0:1789760712.108589:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19722240 -> 19726336 00010000:00010000:2.0:1789760712.108603:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.108616:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.108626:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.108633:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.108638:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.108648:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.108667:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.108675:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1d613 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.108806:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515612928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 91169us (91514us total) trans 4294972132 rc 0/0 00000100:00100000:2.0:1789760712.108817:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4844 00000100:00100000:2.0:1789760712.108827:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690404002904 time 690482899859 state 0x2 ref 0 00000100:00100000:2.0:1789760712.115673:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760712.120044:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613056 00000100:00100000:1.0:1789760712.120061:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613056 00000100:00100000:0.0:1789760712.120191:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690404002904 time 690482899859 state 0x0 ref 1 00000100:00100000:0.0:1789760712.120216:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690483160275 time 690482899859 state 0x1 ref 1 00000100:00100000:0.0:1789760712.120244:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690483160275 time 690585533306 state 0x0 ref 1 00000100:00100000:0.0:1789760712.120257:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515613056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760712.120301:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19726336, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:0.0:1789760712.120309:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.120326:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760712.120331:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.120336:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.120361:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.205747:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972133 is committed 00002000:00010000:0.0:1789760712.205932:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19726336 -> 19730432 00010000:00010000:0.0:1789760712.205948:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.205961:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.205970:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.205976:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760712.205980:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.206000:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.206021:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.206030:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d61a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.206134:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515613056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 85866us (90458us total) trans 4294972133 rc 0/0 00000100:00100000:0.0:1789760712.206144:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4845 00000100:00100000:0.0:1789760712.206154:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690483160275 time 690585533306 state 0x2 ref 0 00000100:00100000:2.0:1789760712.210879:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760712.210967:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613184 00000100:00100000:0.0:1789760712.210978:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613184 00000100:00100000:0.0:1789760712.211043:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690483160275 time 690585533306 state 0x0 ref 1 00000100:00100000:0.0:1789760712.211057:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690585793722 time 690585533306 state 0x1 ref 1 00000100:00100000:0.0:1789760712.211085:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690585793722 time 690676374750 state 0x0 ref 1 00000100:00100000:0.0:1789760712.211097:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515613184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760712.211160:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19730432, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:0.0:1789760712.211169:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.211185:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760712.211189:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.211195:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.211223:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.272759:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972134 is committed 00002000:00010000:3.0:1789760712.273111:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19730432 -> 19734528 00010000:00010000:3.0:1789760712.273140:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.273153:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.273167:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.273175:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760712.273181:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.273190:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.273221:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.273230:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d621 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.273324:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515613184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62223us (62448us total) trans 4294972134 rc 0/0 00000100:00100000:3.0:1789760712.273334:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4846 00000100:00100000:3.0:1789760712.273352:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690585793722 time 690676374750 state 0x2 ref 0 00000100:00100000:3.0:1789760712.281004:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760712.281086:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613312 00000100:00100000:0.0:1789760712.281096:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613312 00000100:00100000:0.0:1789760712.281158:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690585793722 time 690676374750 state 0x0 ref 1 00000100:00100000:0.0:1789760712.281174:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690676635166 time 690676374750 state 0x1 ref 1 00000100:00100000:0.0:1789760712.281216:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690676635166 time 690746505184 state 0x0 ref 1 00000100:00100000:0.0:1789760712.281230:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515613312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760712.281278:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19734528, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:0.0:1789760712.281287:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.281303:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760712.281308:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.281315:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.281330:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.353046:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19734528 -> 19738624 00010000:00010000:3.0:1789760712.353066:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.353085:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.353095:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.353102:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760712.353107:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.353116:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.353135:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.353144:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d628 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.353243:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515613312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 72011us (72244us total) trans 4294972135 rc 0/0 00000100:00100000:3.0:1789760712.353254:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4847 00000100:00100000:3.0:1789760712.353264:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690676635166 time 690746505184 state 0x2 ref 0 00000001:00080000:2.0:1789760712.353552:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972135 is committed 00000100:00100000:2.0:1789760712.358638:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760712.358778:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613440 00000100:00100000:0.0:1789760712.358788:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613440 00000100:00100000:0.0:1789760712.358842:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690676635166 time 690746505184 state 0x0 ref 1 00000100:00100000:0.0:1789760712.358858:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690746765600 time 690746505184 state 0x1 ref 1 00000100:00100000:0.0:1789760712.358887:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690746765600 time 690824176393 state 0x0 ref 1 00000100:00100000:0.0:1789760712.358900:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515613440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760712.358948:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19738624, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:0.0:1789760712.358958:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.358973:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760712.358977:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.358983:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.358998:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.390548:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972136 is committed 00002000:00010000:2.0:1789760712.390915:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19738624 -> 19742720 00010000:00010000:2.0:1789760712.390930:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.390944:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.390953:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.390960:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.390965:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.390974:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.390992:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.391000:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d62f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.391097:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515613440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32194us (32466us total) trans 4294972136 rc 0/0 00000100:00100000:2.0:1789760712.391107:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4848 00000100:00100000:2.0:1789760712.391117:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690746765600 time 690824176393 state 0x2 ref 0 00000100:00100000:2.0:1789760712.394882:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.394959:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613568 00000100:00100000:3.0:1789760712.394968:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613568 00000100:00100000:3.0:1789760712.395012:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690746765600 time 690824176393 state 0x0 ref 1 00000100:00100000:3.0:1789760712.395026:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690824436809 time 690824176393 state 0x1 ref 1 00000100:00100000:3.0:1789760712.395057:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690824436809 time 690860346771 state 0x0 ref 1 00000100:00100000:3.0:1789760712.395071:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515613568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760712.395116:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19742720, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:3.0:1789760712.395126:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.395141:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760712.395146:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.395152:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.395171:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.428795:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972137 is committed 00002000:00010000:3.0:1789760712.428892:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19742720 -> 19746816 00010000:00010000:3.0:1789760712.428908:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.428921:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.428935:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.428942:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760712.428947:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.428956:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.428975:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.428984:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d636 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.429104:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515613568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34032us (34225us total) trans 4294972137 rc 0/0 00000100:00100000:3.0:1789760712.429114:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4849 00000100:00100000:3.0:1789760712.429132:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690824436809 time 690860346771 state 0x2 ref 0 00000100:00100000:3.0:1789760712.432264:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760712.432391:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613696 00000100:00100000:2.0:1789760712.432400:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613696 00000100:00100000:2.0:1789760712.432509:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690824436809 time 690860346771 state 0x0 ref 1 00000100:00100000:2.0:1789760712.432524:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690860607187 time 690860346771 state 0x1 ref 1 00000100:00100000:2.0:1789760712.432546:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690860607187 time 690897834804 state 0x0 ref 1 00000100:00100000:2.0:1789760712.432559:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515613696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760712.432602:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19746816, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:2.0:1789760712.432610:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.432627:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760712.432632:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.432637:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.432650:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.463131:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972138 is committed 00002000:00010000:2.0:1789760712.463190:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19746816 -> 19750912 00010000:00010000:2.0:1789760712.463216:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.463289:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.463301:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.463306:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.463309:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.463315:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.463330:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.463334:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d63d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.463612:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515613696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31050us (31350us total) trans 4294972138 rc 0/0 00000100:00100000:2.0:1789760712.463622:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4850 00000100:00100000:2.0:1789760712.463632:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690860607187 time 690897834804 state 0x2 ref 0 00000100:00100000:2.0:1789760712.466680:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.467153:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613824 00000100:00100000:3.0:1789760712.467164:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613824 00000100:00100000:3.0:1789760712.467229:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690860607187 time 690897834804 state 0x0 ref 1 00000100:00100000:3.0:1789760712.467245:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690898095220 time 690897834804 state 0x1 ref 1 00000100:00100000:3.0:1789760712.467301:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690898095220 time 690932589261 state 0x0 ref 1 00000100:00100000:3.0:1789760712.467313:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515613824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760712.467358:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19750912, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:3.0:1789760712.467367:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.467383:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760712.467388:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.467394:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.467408:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.497268:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19750912 -> 19755008 00010000:00010000:3.0:1789760712.497294:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.497308:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.497317:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.497326:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760712.497331:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.497341:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.497362:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.497370:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d644 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.497845:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515613824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30527us (31167us total) trans 4294972139 rc 0/0 00000100:00100000:3.0:1789760712.497855:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4851 00000100:00100000:3.0:1789760712.497865:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690898095220 time 690932589261 state 0x2 ref 0 00000001:00080000:3.0:1789760712.497904:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972139 is committed 00000100:00100000:0.0:1789760712.500882:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.501015:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515613952 00000100:00100000:3.0:1789760712.501026:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515613952 00000100:00100000:3.0:1789760712.501079:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690898095220 time 690932589261 state 0x0 ref 1 00000100:00100000:3.0:1789760712.501095:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690932849677 time 690932589261 state 0x1 ref 1 00000100:00100000:2.0:1789760712.501235:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690932849677 time 690966521343 state 0x0 ref 1 00000100:00100000:2.0:1789760712.501250:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515613952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760712.501297:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19755008, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:2.0:1789760712.501305:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.501322:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760712.501326:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.501333:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.501346:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760712.535668:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19755008 -> 19759104 00010000:00010000:2.0:1789760712.535689:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.535702:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.535728:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.535735:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.535740:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.535749:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.535771:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.535779:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d64b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.535952:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515613952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34700us (35075us total) trans 4294972140 rc 0/0 00000100:00100000:2.0:1789760712.535963:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4852 00000100:00100000:2.0:1789760712.535972:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690932849677 time 690966521343 state 0x2 ref 0 00000001:00080000:0.0:1789760712.536045:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972140 is committed 00000100:00100000:1.0:1789760712.543255:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.543455:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614080 00000100:00100000:3.0:1789760712.543465:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614080 00000100:00100000:3.0:1789760712.543776:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690932849677 time 690966521343 state 0x0 ref 1 00000100:00100000:3.0:1789760712.543791:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690966781759 time 690966521343 state 0x1 ref 1 00000100:00100000:3.0:1789760712.543822:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690966781759 time 691009110573 state 0x0 ref 1 00000100:00100000:3.0:1789760712.543834:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760712.543882:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19759104, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:3.0:1789760712.543892:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.543908:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760712.543913:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.543920:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.543934:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.601303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972141 is committed 00002000:00010000:2.0:1789760712.601346:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19759104 -> 19763200 00010000:00010000:2.0:1789760712.601363:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.601376:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.601384:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.601391:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.601395:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.601410:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.601429:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.601436:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d652 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.601535:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57698us (58293us total) trans 4294972141 rc 0/0 00000100:00100000:2.0:1789760712.601546:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4853 00000100:00100000:2.0:1789760712.601555:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690966781759 time 691009110573 state 0x2 ref 0 00000100:00100000:2.0:1789760712.607377:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760712.607471:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614208 00000100:00100000:0.0:1789760712.607481:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614208 00000100:00100000:0.0:1789760712.607543:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 690966781759 time 691009110573 state 0x0 ref 1 00000100:00100000:0.0:1789760712.607558:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691009370989 time 691009110573 state 0x1 ref 1 00000100:00100000:0.0:1789760712.607745:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691009370989 time 691073033405 state 0x0 ref 1 00000100:00100000:0.0:1789760712.607759:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760712.607808:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19763200, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:0.0:1789760712.607817:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.607833:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760712.607837:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.607843:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.607856:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760712.651892:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19763200 -> 19767296 00010000:00010000:2.0:1789760712.651913:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.651937:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.651948:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.651962:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.651968:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.651977:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.652244:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.652255:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d659 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.652354:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515614208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44593us (44982us total) trans 4294972142 rc 0/0 00000100:00100000:2.0:1789760712.652365:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4854 00000100:00100000:2.0:1789760712.652374:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691009370989 time 691073033405 state 0x2 ref 0 00000001:00080000:1.0:1789760712.652454:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972142 is committed 00000100:00100000:0.0:1789760712.656654:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.656860:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614336 00000100:00100000:3.0:1789760712.656871:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614336 00000100:00100000:3.0:1789760712.656925:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691009370989 time 691073033405 state 0x0 ref 1 00000100:00100000:3.0:1789760712.656954:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691073293821 time 691073033405 state 0x1 ref 1 00000100:00100000:3.0:1789760712.656980:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691073293821 time 691122269120 state 0x0 ref 1 00000100:00100000:3.0:1789760712.658283:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760712.658350:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19767296, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:3.0:1789760712.658360:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.658376:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760712.658381:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.658388:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.658403:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.713491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972143 is committed 00002000:00010000:0.0:1789760712.713643:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19767296 -> 19771392 00010000:00010000:0.0:1789760712.713659:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.713672:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.713682:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.713696:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760712.713701:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.714108:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.714211:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.714221:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d660 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.714359:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57363us (57710us total) trans 4294972143 rc 0/0 00000100:00100000:0.0:1789760712.714369:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4855 00000100:00100000:0.0:1789760712.714379:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691073293821 time 691122269120 state 0x2 ref 0 00000100:00100000:3.0:1789760712.721582:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760712.721803:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614464 00000100:00100000:2.0:1789760712.721815:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614464 00000100:00100000:2.0:1789760712.721949:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691073293821 time 691122269120 state 0x0 ref 1 00000100:00100000:2.0:1789760712.721964:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691122529536 time 691122269120 state 0x1 ref 1 00000100:00100000:2.0:1789760712.722259:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691122529536 time 691187546210 state 0x0 ref 1 00000100:00100000:2.0:1789760712.722278:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760712.722328:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19771392, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:2.0:1789760712.722337:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.722355:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.722359:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.722365:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.722380:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.768907:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972144 is committed 00002000:00010000:2.0:1789760712.769334:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19771392 -> 19775488 00010000:00010000:2.0:1789760712.769349:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.769360:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.769369:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.769377:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.769382:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.769392:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.769412:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.769419:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d667 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.769506:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47227us (47928us total) trans 4294972144 rc 0/0 00000100:00100000:2.0:1789760712.769516:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4856 00000100:00100000:2.0:1789760712.769526:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691122529536 time 691187546210 state 0x2 ref 0 00000100:00100000:0.0:1789760712.772456:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.772566:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614592 00000100:00100000:3.0:1789760712.772577:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614592 00000100:00100000:3.0:1789760712.772631:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691122529536 time 691187546210 state 0x0 ref 1 00000100:00100000:3.0:1789760712.772648:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691187806626 time 691187546210 state 0x1 ref 1 00000100:00100000:3.0:1789760712.772680:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691187806626 time 691237968263 state 0x0 ref 1 00000100:00100000:3.0:1789760712.772693:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515614592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760712.772783:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19775488, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:3.0:1789760712.772792:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.772808:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760712.772812:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.772818:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.772831:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.828194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972145 is committed 00002000:00010000:2.0:1789760712.828310:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19775488 -> 19779584 00010000:00010000:2.0:1789760712.828325:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.828338:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.828347:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.828355:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760712.828361:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.828370:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.828390:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.828398:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d66e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.828493:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515614592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55797us (56041us total) trans 4294972145 rc 0/0 00000100:00100000:2.0:1789760712.828503:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4857 00000100:00100000:2.0:1789760712.828523:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691187806626 time 691237968263 state 0x2 ref 0 00000100:00100000:0.0:1789760712.838385:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760712.838477:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614720 00000100:00100000:2.0:1789760712.838486:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614720 00000100:00100000:2.0:1789760712.838537:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691187806626 time 691237968263 state 0x0 ref 1 00000100:00100000:2.0:1789760712.838552:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691238228679 time 691237968263 state 0x1 ref 1 00000100:00100000:2.0:1789760712.838581:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691238228679 time 691303870548 state 0x0 ref 1 00000100:00100000:2.0:1789760712.838594:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760712.838638:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19779584, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9807 00010000:00010000:2.0:1789760712.838647:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.838663:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.838668:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.838673:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.838687:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.893428:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972146 is committed 00002000:00010000:0.0:1789760712.893533:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19779584 -> 19783680 00002000:00010000:0.0:1789760712.893539:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760712 -> 1789760713 00002000:00010000:0.0:1789760712.893541:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760712 -> 1789760713 00010000:00010000:0.0:1789760712.893565:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.893579:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.893588:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.893596:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760712.893601:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.893610:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.893630:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.893638:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d675 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.893760:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55163us (55379us total) trans 4294972146 rc 0/0 00000100:00100000:0.0:1789760712.893770:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4858 00000100:00100000:0.0:1789760712.893780:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691238228679 time 691303870548 state 0x2 ref 0 00000100:00100000:0.0:1789760712.903967:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760712.904054:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614848 00000100:00100000:3.0:1789760712.904065:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614848 00000100:00100000:3.0:1789760712.904117:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691238228679 time 691303870548 state 0x0 ref 1 00000100:00100000:3.0:1789760712.904132:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691304130964 time 691303870548 state 0x1 ref 1 00000100:00100000:3.0:1789760712.904161:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691304130964 time 691369449970 state 0x0 ref 1 00000100:00100000:3.0:1789760712.904174:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760712.904235:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19783680, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760712.904243:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.904259:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760712.904263:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.904269:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.904403:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.990731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972147 is committed 00002000:00010000:0.0:1789760712.991150:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19783680 -> 19787776 00010000:00010000:0.0:1789760712.991165:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.991178:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.991187:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.991361:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760712.991367:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.991378:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.991398:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.991406:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d67c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.991501:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 87325us (87540us total) trans 4294972147 rc 0/0 00000100:00100000:0.0:1789760712.991511:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4859 00000100:00100000:0.0:1789760712.991520:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691304130964 time 691369449970 state 0x2 ref 0 00000100:00100000:2.0:1789760712.997548:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760712.997688:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515614976 00000100:00100000:0.0:1789760712.997698:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515614976 00000100:00100000:0.0:1789760712.997835:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691304130964 time 691369449970 state 0x0 ref 1 00000100:00100000:0.0:1789760712.997849:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691369710386 time 691369449970 state 0x1 ref 1 00000100:00100000:0.0:1789760712.998328:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691369710386 time 691463614788 state 0x0 ref 1 00000100:00100000:0.0:1789760712.998356:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760712.998409:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19787776, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760712.998418:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.998443:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760712.998447:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.998453:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.998468:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.023087:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760713.057048:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972148 is committed 00002000:00010000:0.0:1789760713.057125:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19787776 -> 19791872 00010000:00010000:0.0:1789760713.057141:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.057153:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.057162:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.057170:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760713.057175:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.057185:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.057215:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.057223:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d683 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.057328:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515614976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58982us (59784us total) trans 4294972148 rc 0/0 00000100:00100000:0.0:1789760713.057338:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4860 00000100:00100000:0.0:1789760713.057348:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691369710386 time 691463614788 state 0x2 ref 0 00000100:00100000:3.0:1789760713.062225:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.062303:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615104 00000100:00100000:0.0:1789760713.062313:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615104 00000100:00100000:0.0:1789760713.062365:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691369710386 time 691463614788 state 0x0 ref 1 00000100:00100000:0.0:1789760713.062379:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691463875204 time 691463614788 state 0x1 ref 1 00000100:00100000:0.0:1789760713.062424:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691463875204 time 691527713249 state 0x0 ref 1 00000100:00100000:0.0:1789760713.062441:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515615104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.062500:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19791872, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.062511:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.062527:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760713.062532:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.062538:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.062552:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.095219:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972149 is committed 00002000:00010000:0.0:1789760713.095334:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19791872 -> 19795968 00010000:00010000:0.0:1789760713.095350:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.095364:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.095372:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.095380:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760713.095384:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.095391:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.095406:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.095410:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d68a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.095486:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515615104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33045us (33266us total) trans 4294972149 rc 0/0 00000100:00100000:0.0:1789760713.095497:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4861 00000100:00100000:0.0:1789760713.095506:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691463875204 time 691527713249 state 0x2 ref 0 00000100:00100000:3.0:1789760713.100026:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760713.100352:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615232 00000100:00100000:2.0:1789760713.100361:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615232 00000100:00100000:2.0:1789760713.100415:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691463875204 time 691527713249 state 0x0 ref 1 00000100:00100000:2.0:1789760713.100431:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691527973665 time 691527713249 state 0x1 ref 1 00000100:00100000:2.0:1789760713.100460:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691527973665 time 691565749255 state 0x0 ref 1 00000100:00100000:2.0:1789760713.100471:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515615232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760713.100519:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19795968, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:2.0:1789760713.100528:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.100542:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760713.100547:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.100554:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.100568:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.141364:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972150 is committed 00002000:00010000:1.0:1789760713.141394:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19795968 -> 19800064 00010000:00010000:1.0:1789760713.141410:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.141423:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.141433:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.141441:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.141446:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.141455:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.141471:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.141479:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d691 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.141574:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515615232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41100us (41565us total) trans 4294972150 rc 0/0 00000100:00100000:1.0:1789760713.141584:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4862 00000100:00100000:1.0:1789760713.141593:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691527973665 time 691565749255 state 0x2 ref 0 00000100:00100000:2.0:1789760713.150593:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.152147:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615360 00000100:00100000:3.0:1789760713.152160:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615360 00000100:00100000:3.0:1789760713.152226:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691527973665 time 691565749255 state 0x0 ref 1 00000100:00100000:3.0:1789760713.152244:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691566009671 time 691565749255 state 0x1 ref 1 00000100:00100000:3.0:1789760713.152265:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691566009671 time 691617555166 state 0x0 ref 1 00000100:00100000:3.0:1789760713.152278:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515615360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.152330:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19800064, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.152340:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.152355:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760713.152360:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.152366:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.152385:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.196541:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972151 is committed 00002000:00010000:3.0:1789760713.196756:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19800064 -> 19804160 00010000:00010000:3.0:1789760713.196773:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.196786:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.196795:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.196803:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760713.196808:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.196818:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.196846:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.196855:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d698 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.196956:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515615360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44675us (46365us total) trans 4294972151 rc 0/0 00000100:00100000:3.0:1789760713.196966:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4863 00000100:00100000:3.0:1789760713.196975:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691566009671 time 691617555166 state 0x2 ref 0 00000100:00100000:3.0:1789760713.209271:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760713.209435:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615488 00000100:00100000:2.0:1789760713.209447:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615488 00000100:00100000:2.0:1789760713.209503:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691566009671 time 691617555166 state 0x0 ref 1 00000100:00100000:2.0:1789760713.209519:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691617815582 time 691617555166 state 0x1 ref 1 00000100:00100000:2.0:1789760713.209567:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691617815582 time 691674855553 state 0x0 ref 1 00000100:00100000:2.0:1789760713.209587:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515615488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760713.209632:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19804160, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:2.0:1789760713.209641:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.209656:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760713.209661:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.209668:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.209681:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.214145:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544406528:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760713.214232:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760713.214417:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80540b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544406656:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760713.214445:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760713.214476:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406528 00000100:00100000:3.0:1789760713.214483:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406528 00000100:00100000:3.0:1789760713.214525:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406656 00000100:00100000:3.0:1789760713.214527:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406656 00000100:00100000:3.0:1789760713.215216:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789760713.215234:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544406528:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760713.215257:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3838836736 avail 3807379456 00000100:00100000:1.0:1789760713.216227:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:3.0:1789760713.218093:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544406528:12345-0@lo:13:osp-pre-0-0.0 Request processed in 2856us (3853us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760713.218106:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00080000:1.0:1789760713.223902:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760713.223911:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760713.224086:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544406656:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760713.224128:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760713.224223:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544406656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 7986us (9776us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760713.224232:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789760713.224291:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544406528:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760713.230601:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80540b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544406656:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760713.263396:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972152 is committed 00002000:00010000:0.0:1789760713.263696:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19804160 -> 19808256 00010000:00010000:0.0:1789760713.263746:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.263761:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.263780:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.263793:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760713.263798:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.263807:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.263829:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.263846:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d69f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.263937:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515615488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54347us (54670us total) trans 4294972152 rc 0/0 00000100:00100000:0.0:1789760713.263948:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4864 00000100:00100000:0.0:1789760713.263957:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691617815582 time 691674855553 state 0x2 ref 0 00000100:00100000:1.0:1789760713.270334:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.271081:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615616 00000100:00100000:0.0:1789760713.271092:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615616 00000100:00100000:0.0:1789760713.271149:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691617815582 time 691674855553 state 0x0 ref 1 00000100:00100000:0.0:1789760713.271165:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691675115969 time 691674855553 state 0x1 ref 1 00000100:00100000:0.0:1789760713.271199:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691675115969 time 691736480421 state 0x0 ref 1 00000100:00100000:0.0:1789760713.271216:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515615616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.271267:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19808256, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.271276:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.271292:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760713.271297:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.271303:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.271317:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.300405:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19808256 -> 19812352 00000001:00080000:0.0:1789760713.300425:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972153 is committed 00010000:00010000:3.0:1789760713.300427:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.300439:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.300454:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.300462:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760713.300467:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.300476:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.300496:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.300504:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d6a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.300600:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515615616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29382us (30271us total) trans 4294972153 rc 0/0 00000100:00100000:3.0:1789760713.300610:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4865 00000100:00100000:3.0:1789760713.300619:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691675115969 time 691736480421 state 0x2 ref 0 00000100:00100000:2.0:1789760713.303757:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.303955:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615744 00000100:00100000:0.0:1789760713.303965:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615744 00000100:00100000:0.0:1789760713.304020:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691675115969 time 691736480421 state 0x0 ref 1 00000100:00100000:0.0:1789760713.304037:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691736740837 time 691736480421 state 0x1 ref 1 00000100:00100000:0.0:1789760713.304069:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691736740837 time 691769357814 state 0x0 ref 1 00000100:00100000:0.0:1789760713.304082:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515615744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.304132:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19812352, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.304141:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.304158:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760713.304162:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.304169:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.304182:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.329563:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972154 is committed 00002000:00010000:2.0:1789760713.329992:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19812352 -> 19816448 00010000:00010000:2.0:1789760713.330125:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.330139:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.330148:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.330156:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760713.330161:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.330171:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.330191:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.330475:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d6ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.330585:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515615744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26501us (26833us total) trans 4294972154 rc 0/0 00000100:00100000:2.0:1789760713.330595:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4866 00000100:00100000:2.0:1789760713.330604:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691736740837 time 691769357814 state 0x2 ref 0 00000100:00100000:3.0:1789760713.335004:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.335108:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515615872 00000100:00100000:0.0:1789760713.335119:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515615872 00000100:00100000:0.0:1789760713.335171:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691736740837 time 691769357814 state 0x0 ref 1 00000100:00100000:0.0:1789760713.335187:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691769618230 time 691769357814 state 0x1 ref 1 00000100:00100000:0.0:1789760713.335226:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691769618230 time 691800513798 state 0x0 ref 1 00000100:00100000:0.0:1789760713.335238:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515615872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.335395:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19816448, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.335405:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.335422:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760713.335426:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.335432:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.335452:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760713.369534:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972155 is committed 00002000:00010000:0.0:1789760713.370279:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19816448 -> 19820544 00010000:00010000:0.0:1789760713.370296:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.370309:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.370318:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.370326:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760713.370331:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.370392:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.370414:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.370423:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.370576:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515615872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35334us (35576us total) trans 4294972155 rc 0/0 00000100:00100000:0.0:1789760713.370588:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4867 00000100:00100000:0.0:1789760713.370599:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691769618230 time 691800513798 state 0x2 ref 0 00000100:00100000:1.0:1789760713.375283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760713.375433:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616000 00000100:00100000:2.0:1789760713.375443:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616000 00000100:00100000:2.0:1789760713.375497:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691769618230 time 691800513798 state 0x0 ref 1 00000100:00100000:2.0:1789760713.375512:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691800774214 time 691800513798 state 0x1 ref 1 00000100:00100000:2.0:1789760713.375543:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691800774214 time 691840832495 state 0x0 ref 1 00000100:00100000:2.0:1789760713.375565:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515616000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760713.375612:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19820544, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:2.0:1789760713.375621:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.375637:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760713.375642:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.375648:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.375660:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.402579:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972156 is committed 00002000:00010000:0.0:1789760713.402638:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19820544 -> 19824640 00010000:00010000:0.0:1789760713.402654:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.402667:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.402677:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.402684:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760713.402689:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.402699:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.402741:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.402754:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1d6bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.402843:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515616000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27285us (27564us total) trans 4294972156 rc 0/0 00000100:00100000:0.0:1789760713.402853:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4868 00000100:00100000:0.0:1789760713.402862:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691800774214 time 691840832495 state 0x2 ref 0 00000100:00100000:2.0:1789760713.408675:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.408951:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616128 00000100:00100000:3.0:1789760713.408962:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616128 00000100:00100000:3.0:1789760713.409088:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691800774214 time 691840832495 state 0x0 ref 1 00000100:00100000:3.0:1789760713.409106:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691841092911 time 691840832495 state 0x1 ref 1 00000100:00100000:3.0:1789760713.409312:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691841092911 time 691874599019 state 0x0 ref 1 00000100:00100000:3.0:1789760713.409327:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515616128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.409377:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19824640, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.409387:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.409403:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760713.409408:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.409415:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.409430:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.452730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972157 is committed 00002000:00010000:2.0:1789760713.452925:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19824640 -> 19828736 00010000:00010000:2.0:1789760713.452940:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.452952:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.452961:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.452969:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760713.452974:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.452983:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.453002:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.453010:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d6c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.453105:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515616128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43777us (44434us total) trans 4294972157 rc 0/0 00000100:00100000:2.0:1789760713.453180:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4869 00000100:00100000:2.0:1789760713.453190:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691841092911 time 691874599019 state 0x2 ref 0 00000100:00100000:2.0:1789760713.461383:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.461468:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616256 00000100:00100000:0.0:1789760713.461477:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616256 00000100:00100000:0.0:1789760713.461532:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691841092911 time 691874599019 state 0x0 ref 1 00000100:00100000:0.0:1789760713.461546:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691874859435 time 691874599019 state 0x1 ref 1 00000100:00100000:0.0:1789760713.461575:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691874859435 time 691926864525 state 0x0 ref 1 00000100:00100000:0.0:1789760713.461588:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515616256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.461643:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19828736, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.461652:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.461668:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760713.461672:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.461677:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.461692:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.490239:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972158 is committed 00002000:00010000:1.0:1789760713.490247:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19828736 -> 19832832 00010000:00010000:1.0:1789760713.490261:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.490274:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.490284:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.490291:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.490295:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.490303:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.490325:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.490334:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d6c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.490435:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515616256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28843us (29056us total) trans 4294972158 rc 0/0 00000100:00100000:1.0:1789760713.490445:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4870 00000100:00100000:1.0:1789760713.490455:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691874859435 time 691926864525 state 0x2 ref 0 00000100:00100000:1.0:1789760713.495509:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.495627:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616384 00000100:00100000:3.0:1789760713.495639:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616384 00000100:00100000:3.0:1789760713.495694:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691874859435 time 691926864525 state 0x0 ref 1 00000100:00100000:3.0:1789760713.495754:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691927124941 time 691926864525 state 0x1 ref 1 00000100:00100000:3.0:1789760713.495775:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691927124941 time 691961064895 state 0x0 ref 1 00000100:00100000:3.0:1789760713.495789:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.495849:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19832832, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.495859:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.495874:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760713.495879:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.495885:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.495898:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.527867:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19832832 -> 19836928 00010000:00010000:3.0:1789760713.527887:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.527899:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.527909:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.527916:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760713.527920:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.527930:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.527948:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.527956:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.528045:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515616384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32253us (32543us total) trans 4294972159 rc 0/0 00000100:00100000:3.0:1789760713.528056:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4871 00000100:00100000:3.0:1789760713.528066:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691927124941 time 691961064895 state 0x2 ref 0 00000100:00100000:2.0:1789760713.531416:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.531534:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616512 00000100:00100000:3.0:1789760713.531545:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616512 00000100:00100000:3.0:1789760713.531597:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691927124941 time 691961064895 state 0x0 ref 1 00000100:00100000:3.0:1789760713.531613:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691961325311 time 691961064895 state 0x1 ref 1 00000100:00100000:3.0:1789760713.531643:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691961325311 time 691996931977 state 0x0 ref 1 00000100:00100000:3.0:1789760713.531655:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515616512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.531702:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19836928, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.531744:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.531760:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760713.531765:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.531771:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.531785:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.531940:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972159 is committed 00002000:00010000:3.0:1789760713.573142:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19836928 -> 19841024 00010000:00010000:3.0:1789760713.573163:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.573176:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.573184:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.573200:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760713.573205:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.573215:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.573237:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.573244:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.573325:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515616512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41668us (41915us total) trans 4294972160 rc 0/0 00000100:00100000:3.0:1789760713.573335:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4872 00000100:00100000:3.0:1789760713.573344:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691961325311 time 691996931977 state 0x2 ref 0 00000001:00080000:3.0:1789760713.573380:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972160 is committed 00000100:00100000:3.0:1789760713.576635:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760713.576740:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616640 00000100:00100000:1.0:1789760713.576751:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616640 00000100:00100000:1.0:1789760713.576808:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691961325311 time 691996931977 state 0x0 ref 1 00000100:00100000:1.0:1789760713.576824:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691997192393 time 691996931977 state 0x1 ref 1 00000100:00100000:1.0:1789760713.576857:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691997192393 time 692042146036 state 0x0 ref 1 00000100:00100000:1.0:1789760713.576869:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760713.576915:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19841024, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:1.0:1789760713.576924:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.576939:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.576944:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.576950:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.576964:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.631257:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972161 is committed 00002000:00010000:1.0:1789760713.631282:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19841024 -> 19845120 00010000:00010000:1.0:1789760713.631296:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.631309:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.631318:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.631325:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.631330:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.631348:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.631369:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.631377:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d6de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.631877:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55003us (55245us total) trans 4294972161 rc 0/0 00000100:00100000:1.0:1789760713.631888:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4873 00000100:00100000:1.0:1789760713.631898:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691997192393 time 692042146036 state 0x2 ref 0 00000100:00100000:3.0:1789760713.639140:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.639662:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616768 00000100:00100000:0.0:1789760713.639672:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616768 00000100:00100000:0.0:1789760713.639755:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 691997192393 time 692042146036 state 0x0 ref 1 00000100:00100000:0.0:1789760713.639865:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692042406452 time 692042146036 state 0x1 ref 1 00000100:00100000:0.0:1789760713.639897:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692042406452 time 692105186280 state 0x0 ref 1 00000100:00100000:0.0:1789760713.639909:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.639957:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19845120, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.639966:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.639983:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.639988:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.639995:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.640010:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760713.681137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972162 is committed 00002000:00010000:2.0:1789760713.681306:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19845120 -> 19849216 00010000:00010000:2.0:1789760713.681321:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.681333:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.681440:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.681450:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760713.681455:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.681464:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.681490:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.681498:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d6e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.681689:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41777us (42552us total) trans 4294972162 rc 0/0 00000100:00100000:2.0:1789760713.681700:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4874 00000100:00100000:2.0:1789760713.681744:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692042406452 time 692105186280 state 0x2 ref 0 00000100:00100000:2.0:1789760713.685326:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.685419:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515616896 00000100:00100000:3.0:1789760713.685428:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515616896 00000100:00100000:3.0:1789760713.685486:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692042406452 time 692105186280 state 0x0 ref 1 00000100:00100000:3.0:1789760713.685502:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692105446696 time 692105186280 state 0x1 ref 1 00000100:00100000:3.0:1789760713.685634:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692105446696 time 692150921439 state 0x0 ref 1 00000100:00100000:3.0:1789760713.685648:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.685696:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19849216, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.685768:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.685785:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.685789:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.685796:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.685811:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.757220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972163 is committed 00002000:00010000:1.0:1789760713.757321:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19849216 -> 19853312 00010000:00010000:1.0:1789760713.757336:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.757350:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.757365:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.757373:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.757377:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.757386:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.757405:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.757412:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d6ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.757509:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515616896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71859us (72187us total) trans 4294972163 rc 0/0 00000100:00100000:1.0:1789760713.757518:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4875 00000100:00100000:1.0:1789760713.757528:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692105446696 time 692150921439 state 0x2 ref 0 00000100:00100000:2.0:1789760713.763220:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.763316:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617024 00000100:00100000:3.0:1789760713.763327:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617024 00000100:00100000:3.0:1789760713.763385:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692105446696 time 692150921439 state 0x0 ref 1 00000100:00100000:3.0:1789760713.763403:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692151181855 time 692150921439 state 0x1 ref 1 00000100:00100000:3.0:1789760713.763441:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692151181855 time 692228729071 state 0x0 ref 1 00000100:00100000:3.0:1789760713.763456:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.763520:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19853312, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.763530:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.763546:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.763561:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.763568:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.763583:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.797091:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972164 is committed 00002000:00010000:3.0:1789760713.797317:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19853312 -> 19857408 00010000:00010000:3.0:1789760713.797334:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.797346:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.797355:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.797362:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760713.797366:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.797374:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.797393:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.797401:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d6f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.797490:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34035us (34276us total) trans 4294972164 rc 0/0 00000100:00100000:3.0:1789760713.797500:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4876 00000100:00100000:3.0:1789760713.797510:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692151181855 time 692228729071 state 0x2 ref 0 00000100:00100000:0.0:1789760713.801200:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.801375:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617152 00000100:00100000:3.0:1789760713.801386:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617152 00000100:00100000:3.0:1789760713.801433:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692151181855 time 692228729071 state 0x0 ref 1 00000100:00100000:3.0:1789760713.801448:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692228989487 time 692228729071 state 0x1 ref 1 00000100:00100000:3.0:1789760713.801478:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692228989487 time 692266767497 state 0x0 ref 1 00000100:00100000:3.0:1789760713.801489:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515617152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.801538:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19857408, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:3.0:1789760713.801548:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.801566:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.801570:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.801576:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.801589:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.829300:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972165 is committed 00002000:00010000:1.0:1789760713.829451:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19857408 -> 19861504 00010000:00010000:1.0:1789760713.829468:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.829486:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.829495:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.829502:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.829507:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.829517:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.829536:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.829544:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d6fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.829637:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515617152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28144us (28451us total) trans 4294972165 rc 0/0 00000100:00100000:1.0:1789760713.829647:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4877 00000100:00100000:1.0:1789760713.829657:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692228989487 time 692266767497 state 0x2 ref 0 00000100:00100000:2.0:1789760713.832487:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.832611:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617280 00000100:00100000:0.0:1789760713.832622:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617280 00000100:00100000:0.0:1789760713.832676:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692228989487 time 692266767497 state 0x0 ref 1 00000100:00100000:0.0:1789760713.832692:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692267027913 time 692266767497 state 0x1 ref 1 00000100:00100000:0.0:1789760713.832790:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692267027913 time 692298078626 state 0x0 ref 1 00000100:00100000:0.0:1789760713.832802:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515617280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.832858:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19861504, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9807 00010000:00010000:0.0:1789760713.832867:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.832883:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.832888:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.832894:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.832907:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.870452:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19861504 -> 19865600 00002000:00010000:3.0:1789760713.870461:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760713 -> 1789760714 00002000:00010000:3.0:1789760713.870464:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760713 -> 1789760714 00010000:00010000:3.0:1789760713.870490:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.870508:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.870517:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.870523:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760713.870527:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.870535:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.870554:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.870562:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d701 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.870654:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515617280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37849us (38171us total) trans 4294972166 rc 0/0 00000100:00100000:3.0:1789760713.870665:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4878 00000100:00100000:3.0:1789760713.870675:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692267027913 time 692298078626 state 0x2 ref 0 00000001:00080000:1.0:1789760713.873751:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972166 is committed 00000100:00100000:2.0:1789760713.878584:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760713.878671:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617408 00000100:00100000:1.0:1789760713.878680:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617408 00000100:00100000:1.0:1789760713.878842:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692267027913 time 692298078626 state 0x0 ref 1 00000100:00100000:1.0:1789760713.878857:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692298339042 time 692298078626 state 0x1 ref 1 00000100:00100000:1.0:1789760713.878918:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692298339042 time 692344207187 state 0x0 ref 1 00000100:00100000:1.0:1789760713.878931:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760713.878981:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19865600, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:1.0:1789760713.878991:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.879009:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.879014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.879020:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.879035:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.905391:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972167 is committed 00002000:00010000:1.0:1789760713.905583:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19865600 -> 19869696 00010000:00010000:1.0:1789760713.905598:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.905611:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.905620:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.905628:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.905632:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.905641:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.905661:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.905668:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d708 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.905774:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26840us (27195us total) trans 4294972167 rc 0/0 00000100:00100000:1.0:1789760713.905783:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4879 00000100:00100000:1.0:1789760713.905793:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692298339042 time 692344207187 state 0x2 ref 0 00000100:00100000:0.0:1789760713.908691:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760713.908770:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617536 00000100:00100000:1.0:1789760713.908780:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617536 00000100:00100000:1.0:1789760713.908835:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692298339042 time 692344207187 state 0x0 ref 1 00000100:00100000:1.0:1789760713.908864:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692344467603 time 692344207187 state 0x1 ref 1 00000100:00100000:1.0:1789760713.908884:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692344467603 time 692374174404 state 0x0 ref 1 00000100:00100000:1.0:1789760713.908896:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515617536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760713.908942:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19869696, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:1.0:1789760713.908951:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.908965:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.908969:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.908975:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.908988:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.942057:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19869696 -> 19873792 00010000:00010000:1.0:1789760713.942079:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.942093:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.942104:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.942123:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.942128:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.942138:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.942158:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.942167:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d70f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.942266:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515617536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33367us (33579us total) trans 4294972168 rc 0/0 00000100:00100000:1.0:1789760713.942277:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4880 00000100:00100000:1.0:1789760713.942286:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692344467603 time 692374174404 state 0x2 ref 0 00000001:00080000:2.0:1789760713.942412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972168 is committed 00000100:00100000:1.0:1789760713.945866:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760713.946019:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617664 00000100:00100000:3.0:1789760713.946029:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617664 00000100:00100000:3.0:1789760713.946084:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692344467603 time 692374174404 state 0x0 ref 1 00000100:00100000:3.0:1789760713.946098:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692374434820 time 692374174404 state 0x1 ref 1 00000100:00100000:3.0:1789760713.946135:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692374434820 time 692411423972 state 0x0 ref 1 00000100:00100000:3.0:1789760713.946149:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760713.946208:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19873792, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:3.0:1789760713.946218:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.946234:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760713.946238:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.946244:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.946259:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.970545:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19873792 -> 19877888 00010000:00010000:1.0:1789760713.970566:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970579:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970589:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970603:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760713.970609:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970618:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970639:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.970647:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1d716 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.970772:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515617664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24620us (24909us total) trans 4294972169 rc 0/0 00000100:00100000:1.0:1789760713.970784:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4881 00000100:00100000:1.0:1789760713.970793:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692374434820 time 692411423972 state 0x2 ref 0 00000001:00080000:0.0:1789760713.970975:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972169 is committed 00000100:00100000:3.0:1789760713.975057:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760713.975203:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617792 00000100:00100000:0.0:1789760713.975213:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617792 00000100:00100000:0.0:1789760713.975261:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692374434820 time 692411423972 state 0x0 ref 1 00000100:00100000:0.0:1789760713.975275:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692411684388 time 692411423972 state 0x1 ref 1 00000100:00100000:0.0:1789760713.975305:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692411684388 time 692440594300 state 0x0 ref 1 00000100:00100000:0.0:1789760713.975316:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760713.975363:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19877888, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:0.0:1789760713.975372:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.975387:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.975391:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.975397:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.975412:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.014407:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19877888 -> 19881984 00010000:00010000:0.0:1789760714.014428:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.014441:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.014454:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.014460:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760714.014465:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.014473:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.014491:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.014498:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d71d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.014592:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515617792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39272us (39540us total) trans 4294972170 rc 0/0 00000100:00100000:0.0:1789760714.014602:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4882 00000100:00100000:0.0:1789760714.014613:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692411684388 time 692440594300 state 0x2 ref 0 00000001:00080000:1.0:1789760714.014684:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972170 is committed 00000100:00100000:0.0:1789760714.019379:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.019600:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515617920 00000100:00100000:3.0:1789760714.019619:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515617920 00000100:00100000:3.0:1789760714.019684:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692411684388 time 692440594300 state 0x0 ref 1 00000100:00100000:3.0:1789760714.019699:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692440854716 time 692440594300 state 0x1 ref 1 00000100:00100000:3.0:1789760714.019746:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692440854716 time 692485035570 state 0x0 ref 1 00000100:00100000:3.0:1789760714.019770:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.019819:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19881984, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:3.0:1789760714.019828:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.019853:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760714.019857:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.019863:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.019877:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760714.020402:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760714.046135:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760714.049609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972171 is committed 00002000:00010000:3.0:1789760714.049744:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19881984 -> 19886080 00010000:00010000:3.0:1789760714.049769:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.049784:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.049794:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.049803:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.049808:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.049819:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.049848:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.049857:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d724 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.049954:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515617920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30191us (30578us total) trans 4294972171 rc 0/0 00000100:00100000:3.0:1789760714.049964:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4883 00000100:00100000:3.0:1789760714.049974:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692440854716 time 692485035570 state 0x2 ref 0 00000100:00100000:0.0:1789760714.053435:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.053514:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618048 00000100:00100000:3.0:1789760714.053523:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618048 00000100:00100000:3.0:1789760714.053577:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692440854716 time 692485035570 state 0x0 ref 1 00000100:00100000:3.0:1789760714.053592:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692485295986 time 692485035570 state 0x1 ref 1 00000100:00100000:3.0:1789760714.053625:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692485295986 time 692518913642 state 0x0 ref 1 00000100:00100000:3.0:1789760714.053636:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515618048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.053681:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19886080, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:3.0:1789760714.053690:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.053728:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760714.053733:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.053739:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.053753:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.102482:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19886080 -> 19890176 00010000:00010000:3.0:1789760714.102503:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.102518:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.102529:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.102538:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.102543:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.102554:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.102575:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.102584:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d72b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.102677:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515618048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49036us (49247us total) trans 4294972172 rc 0/0 00000100:00100000:3.0:1789760714.102687:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4884 00000100:00100000:3.0:1789760714.102696:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692485295986 time 692518913642 state 0x2 ref 0 00000001:00080000:2.0:1789760714.103865:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972172 is committed 00000100:00100000:0.0:1789760714.106326:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.106428:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618176 00000100:00100000:2.0:1789760714.106439:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618176 00000100:00100000:2.0:1789760714.106488:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692485295986 time 692518913642 state 0x0 ref 1 00000100:00100000:2.0:1789760714.106504:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692519174058 time 692518913642 state 0x1 ref 1 00000100:00100000:2.0:1789760714.106533:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692519174058 time 692571822671 state 0x0 ref 1 00000100:00100000:2.0:1789760714.106545:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.106597:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19890176, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:2.0:1789760714.106605:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.106622:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760714.106627:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.106633:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.106649:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.156508:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972173 is committed 00002000:00010000:2.0:1789760714.156592:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19890176 -> 19894272 00010000:00010000:2.0:1789760714.156609:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.156621:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.156631:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.156637:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.156642:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.156651:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.156673:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.156681:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d732 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.157086:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50528us (50755us total) trans 4294972173 rc 0/0 00000100:00100000:2.0:1789760714.157097:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4885 00000100:00100000:2.0:1789760714.157232:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692519174058 time 692571822671 state 0x2 ref 0 00000100:00100000:0.0:1789760714.162067:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.162643:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618304 00000100:00100000:2.0:1789760714.162654:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618304 00000100:00100000:2.0:1789760714.163196:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692519174058 time 692571822671 state 0x0 ref 1 00000100:00100000:2.0:1789760714.163215:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692572083087 time 692571822671 state 0x1 ref 1 00000100:00100000:2.0:1789760714.163247:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692572083087 time 692628535442 state 0x0 ref 1 00000100:00100000:2.0:1789760714.163261:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515618304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.163591:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19894272, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:2.0:1789760714.163602:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.163616:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760714.163621:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.163626:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.163640:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.211567:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19894272 -> 19898368 00010000:00010000:1.0:1789760714.211585:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.211598:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.211607:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.211614:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760714.211619:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.211629:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.211668:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.211676:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d739 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.212065:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515618304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48803us (50002us total) trans 4294972174 rc 0/0 00000100:00100000:1.0:1789760714.212549:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4886 00000100:00100000:1.0:1789760714.212563:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692572083087 time 692628535442 state 0x2 ref 0 00000001:00080000:3.0:1789760714.212821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972174 is committed 00000100:00100000:0.0:1789760714.216980:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760714.217060:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618432 00000100:00100000:1.0:1789760714.217070:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618432 00000100:00100000:1.0:1789760714.217127:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692572083087 time 692628535442 state 0x0 ref 1 00000100:00100000:1.0:1789760714.217145:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692628795858 time 692628535442 state 0x1 ref 1 00000100:00100000:1.0:1789760714.217176:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692628795858 time 692682464791 state 0x0 ref 1 00000100:00100000:1.0:1789760714.217199:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760714.217249:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19898368, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:1.0:1789760714.217258:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217274:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760714.217279:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.217285:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217299:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.249634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972175 is committed 00002000:00010000:1.0:1789760714.249857:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19898368 -> 19902464 00010000:00010000:1.0:1789760714.249874:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249887:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249897:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249904:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760714.249910:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249920:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249939:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.249947:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d740 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.250029:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32839us (33056us total) trans 4294972175 rc 0/0 00000100:00100000:1.0:1789760714.250040:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4887 00000100:00100000:1.0:1789760714.250050:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692628795858 time 692682464791 state 0x2 ref 0 00000100:00100000:0.0:1789760714.254451:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.254528:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618560 00000100:00100000:3.0:1789760714.254538:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618560 00000100:00100000:3.0:1789760714.254673:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692628795858 time 692682464791 state 0x0 ref 1 00000100:00100000:3.0:1789760714.254689:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692682725207 time 692682464791 state 0x1 ref 1 00000100:00100000:3.0:1789760714.254864:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692682725207 time 692720151741 state 0x0 ref 1 00000100:00100000:3.0:1789760714.254877:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.254928:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19902464, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:3.0:1789760714.254937:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.254953:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760714.254957:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.254964:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.254979:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.322800:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19902464 -> 19906560 00010000:00010000:0.0:1789760714.322820:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.322833:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.322843:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.322851:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760714.322856:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.322865:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.322885:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.322893:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1d747 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.322984:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515618560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 68104us (68536us total) trans 4294972176 rc 0/0 00000100:00100000:0.0:1789760714.322995:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4888 00000100:00100000:0.0:1789760714.323004:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692682725207 time 692720151741 state 0x2 ref 0 00000001:00080000:0.0:1789760714.323039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972176 is committed 00000100:00100000:0.0:1789760714.327958:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.328039:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618688 00000100:00100000:3.0:1789760714.328048:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618688 00000100:00100000:3.0:1789760714.328106:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692682725207 time 692720151741 state 0x0 ref 1 00000100:00100000:3.0:1789760714.328131:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692720412157 time 692720151741 state 0x1 ref 1 00000100:00100000:3.0:1789760714.328152:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692720412157 time 692793441812 state 0x0 ref 1 00000100:00100000:3.0:1789760714.328164:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.328220:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19906560, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:3.0:1789760714.328229:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.328245:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760714.328250:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.328257:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.328270:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.361081:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972177 is committed 00002000:00010000:0.0:1789760714.361244:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19906560 -> 19910656 00010000:00010000:0.0:1789760714.361259:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.361271:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.361280:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.361287:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760714.361292:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.361301:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.361319:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.361327:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1d74e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.361453:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33287us (33499us total) trans 4294972177 rc 0/0 00000100:00100000:0.0:1789760714.361464:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4889 00000100:00100000:0.0:1789760714.361474:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692720412157 time 692793441812 state 0x2 ref 0 00000100:00100000:3.0:1789760714.365397:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760714.366942:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618816 00000100:00100000:0.0:1789760714.366954:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618816 00000100:00100000:0.0:1789760714.367009:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692720412157 time 692793441812 state 0x0 ref 1 00000100:00100000:0.0:1789760714.367023:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692793702228 time 692793441812 state 0x1 ref 1 00000100:00100000:1.0:1789760714.367060:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692793702228 time 692832347308 state 0x0 ref 1 00000100:00100000:1.0:1789760714.367075:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760714.367128:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19910656, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:1.0:1789760714.367137:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.367152:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760714.367157:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.367164:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.367177:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.402006:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972178 is committed 00002000:00010000:2.0:1789760714.402111:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19910656 -> 19914752 00010000:00010000:2.0:1789760714.402127:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.402141:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.402150:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.402158:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.402162:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.402172:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.402207:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.402216:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d755 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.402306:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515618816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35229us (36913us total) trans 4294972178 rc 0/0 00000100:00100000:2.0:1789760714.402317:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4890 00000100:00100000:2.0:1789760714.402328:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692793702228 time 692832347308 state 0x2 ref 0 00000100:00100000:3.0:1789760714.406412:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.406787:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515618944 00000100:00100000:2.0:1789760714.406797:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515618944 00000100:00100000:2.0:1789760714.406859:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692793702228 time 692832347308 state 0x0 ref 1 00000100:00100000:2.0:1789760714.406874:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692832607724 time 692832347308 state 0x1 ref 1 00000100:00100000:2.0:1789760714.406894:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692832607724 time 692872183656 state 0x0 ref 1 00000100:00100000:2.0:1789760714.406905:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515618944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.406950:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19914752, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:2.0:1789760714.406959:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.406974:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760714.406979:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.406985:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.406997:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.435394:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972179 is committed 00002000:00010000:1.0:1789760714.435492:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19914752 -> 19918848 00010000:00010000:1.0:1789760714.435508:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.435521:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.435530:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.435537:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760714.435542:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.435550:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.435570:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.435577:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d75c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.435664:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515618944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28754us (29259us total) trans 4294972179 rc 0/0 00000100:00100000:1.0:1789760714.435675:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4891 00000100:00100000:1.0:1789760714.435685:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692832607724 time 692872183656 state 0x2 ref 0 00000100:00100000:0.0:1789760714.440001:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760714.440075:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619072 00000100:00100000:1.0:1789760714.440086:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619072 00000100:00100000:1.0:1789760714.440145:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692832607724 time 692872183656 state 0x0 ref 1 00000100:00100000:1.0:1789760714.440170:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692872444072 time 692872183656 state 0x1 ref 1 00000100:00100000:1.0:1789760714.440206:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692872444072 time 692905495321 state 0x0 ref 1 00000100:00100000:1.0:1789760714.440219:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515619072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760714.440267:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19918848, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:1.0:1789760714.440276:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.440292:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760714.440296:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.440301:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.440313:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.467229:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972180 is committed 00002000:00010000:2.0:1789760714.467308:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19918848 -> 19922944 00010000:00010000:2.0:1789760714.467324:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.467336:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.467346:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.467354:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.467360:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.467369:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.467389:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.467396:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d763 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.467523:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515619072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27301us (27527us total) trans 4294972180 rc 0/0 00000100:00100000:2.0:1789760714.467533:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4892 00000100:00100000:2.0:1789760714.467542:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692872444072 time 692905495321 state 0x2 ref 0 00000100:00100000:3.0:1789760714.470661:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760714.470743:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619200 00000100:00100000:1.0:1789760714.470753:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619200 00000100:00100000:1.0:1789760714.470808:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692872444072 time 692905495321 state 0x0 ref 1 00000100:00100000:1.0:1789760714.470825:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692905755737 time 692905495321 state 0x1 ref 1 00000100:00100000:1.0:1789760714.470860:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692905755737 time 692936149056 state 0x0 ref 1 00000100:00100000:1.0:1789760714.470871:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760714.470919:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19922944, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9807 00010000:00010000:1.0:1789760714.470927:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.470942:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760714.470946:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.470952:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.470965:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.500733:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972181 is committed 00002000:00010000:2.0:1789760714.500886:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19922944 -> 19927040 00002000:00010000:2.0:1789760714.500891:0:6711:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:2.0:1789760714.500904:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.500916:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.500925:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.500932:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.500937:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.500946:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.500964:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.500977:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d76a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.501066:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30191us (30410us total) trans 4294972181 rc 0/0 00000100:00100000:2.0:1789760714.501076:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4893 00000100:00100000:2.0:1789760714.501085:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692905755737 time 692936149056 state 0x2 ref 0 00000100:00100000:0.0:1789760714.505605:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.505911:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619328 00000100:00100000:3.0:1789760714.505926:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619328 00000100:00100000:3.0:1789760714.505980:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692905755737 time 692936149056 state 0x0 ref 1 00000100:00100000:3.0:1789760714.505995:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692936409472 time 692936149056 state 0x1 ref 1 00000100:00100000:3.0:1789760714.506069:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692936409472 time 692971357403 state 0x0 ref 1 00000100:00100000:3.0:1789760714.506082:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.506138:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19927040, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:3.0:1789760714.506147:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.506162:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760714.506166:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.506172:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.506198:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.537319:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972182 is committed 00002000:00010000:3.0:1789760714.537407:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19927040 -> 19931136 00010000:00010000:3.0:1789760714.537422:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.537435:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.537444:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.537452:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.537456:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.537465:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.537484:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.537492:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d771 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.537844:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31759us (32241us total) trans 4294972182 rc 0/0 00000100:00100000:3.0:1789760714.537859:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4894 00000100:00100000:3.0:1789760714.537869:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692936409472 time 692971357403 state 0x2 ref 0 00000100:00100000:0.0:1789760714.541032:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.541116:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619456 00000100:00100000:3.0:1789760714.541126:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619456 00000100:00100000:3.0:1789760714.541178:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692936409472 time 692971357403 state 0x0 ref 1 00000100:00100000:3.0:1789760714.541204:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692971617819 time 692971357403 state 0x1 ref 1 00000100:00100000:3.0:1789760714.541236:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692971617819 time 693006525422 state 0x0 ref 1 00000100:00100000:3.0:1789760714.541249:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.541293:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19931136, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:3.0:1789760714.541302:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.541318:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760714.541322:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.541328:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.541341:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.569137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972183 is committed 00002000:00010000:3.0:1789760714.569249:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19931136 -> 19935232 00010000:00010000:3.0:1789760714.569265:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.569278:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.569288:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.569295:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.569299:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.569308:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.569328:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.569336:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d778 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.569438:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28187us (28411us total) trans 4294972183 rc 0/0 00000100:00100000:3.0:1789760714.569449:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4895 00000100:00100000:3.0:1789760714.569459:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692971617819 time 693006525422 state 0x2 ref 0 00000100:00100000:0.0:1789760714.572985:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.573445:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619584 00000100:00100000:2.0:1789760714.573454:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619584 00000100:00100000:2.0:1789760714.573503:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 692971617819 time 693006525422 state 0x0 ref 1 00000100:00100000:2.0:1789760714.573518:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693006785838 time 693006525422 state 0x1 ref 1 00000100:00100000:2.0:1789760714.573548:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693006785838 time 693038837911 state 0x0 ref 1 00000100:00100000:2.0:1789760714.573559:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515619584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.573602:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19935232, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:2.0:1789760714.573609:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.573624:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.573628:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.573635:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.573649:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.611767:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19935232 -> 19939328 00010000:00010000:2.0:1789760714.611788:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.611802:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.611812:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.611820:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.611836:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.611846:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.611865:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.611872:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d77f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.611958:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515619584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38396us (38978us total) trans 4294972184 rc 0/0 00000100:00100000:2.0:1789760714.611969:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4896 00000100:00100000:2.0:1789760714.611979:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693006785838 time 693038837911 state 0x2 ref 0 00000001:00080000:3.0:1789760714.613117:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972184 is committed 00000100:00100000:3.0:1789760714.615276:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760714.615509:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619712 00000100:00100000:1.0:1789760714.615519:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619712 00000100:00100000:1.0:1789760714.615568:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693006785838 time 693038837911 state 0x0 ref 1 00000100:00100000:1.0:1789760714.615583:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693039098327 time 693038837911 state 0x1 ref 1 00000100:00100000:1.0:1789760714.615610:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693039098327 time 693080900151 state 0x0 ref 1 00000100:00100000:1.0:1789760714.615622:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760714.615669:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19939328, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:1.0:1789760714.615678:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.615694:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.615699:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.615743:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.615759:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.652490:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972185 is committed 00002000:00010000:2.0:1789760714.652605:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19939328 -> 19943424 00010000:00010000:2.0:1789760714.652620:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.652633:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.652643:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.652650:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.652654:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.652663:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.652682:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.652695:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d786 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.652808:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37183us (37536us total) trans 4294972185 rc 0/0 00000100:00100000:2.0:1789760714.652819:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4897 00000100:00100000:2.0:1789760714.652840:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693039098327 time 693080900151 state 0x2 ref 0 00000100:00100000:0.0:1789760714.656259:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.656379:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619840 00000100:00100000:2.0:1789760714.656390:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619840 00000100:00100000:2.0:1789760714.656442:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693039098327 time 693080900151 state 0x0 ref 1 00000100:00100000:2.0:1789760714.656458:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693081160567 time 693080900151 state 0x1 ref 1 00000100:00100000:2.0:1789760714.656489:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693081160567 time 693121778411 state 0x0 ref 1 00000100:00100000:2.0:1789760714.656501:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515619840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.656547:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19943424, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:2.0:1789760714.656561:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.656575:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.656579:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.656585:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.656602:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.690662:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19943424 -> 19947520 00010000:00010000:0.0:1789760714.690681:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.690695:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.690727:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.690734:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760714.690739:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.690748:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.690766:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.690774:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1d78d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.690866:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515619840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34362us (34612us total) trans 4294972186 rc 0/0 00000100:00100000:0.0:1789760714.690876:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4898 00000100:00100000:0.0:1789760714.690885:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693081160567 time 693121778411 state 0x2 ref 0 00000001:00080000:0.0:1789760714.690921:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972186 is committed 00000100:00100000:0.0:1789760714.697746:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.699552:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515619968 00000100:00100000:3.0:1789760714.699565:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515619968 00000100:00100000:3.0:1789760714.699752:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693081160567 time 693121778411 state 0x0 ref 1 00000100:00100000:3.0:1789760714.699928:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693122038827 time 693121778411 state 0x1 ref 1 00000100:00100000:3.0:1789760714.699960:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693122038827 time 693165249871 state 0x0 ref 1 00000100:00100000:3.0:1789760714.699975:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515619968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.700023:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19947520, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:3.0:1789760714.700032:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.700048:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760714.700052:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.700059:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.700138:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.747767:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19947520 -> 19951616 00010000:00010000:2.0:1789760714.747786:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.747798:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.747807:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.747813:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760714.747818:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.747833:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.747858:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.747866:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1d794 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.747958:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515619968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47982us (50218us total) trans 4294972187 rc 0/0 00000100:00100000:2.0:1789760714.747968:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4899 00000100:00100000:2.0:1789760714.747978:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693122038827 time 693165249871 state 0x2 ref 0 00000001:00080000:0.0:1789760714.751656:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972187 is committed 00000100:00100000:2.0:1789760714.752423:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.752496:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620096 00000100:00100000:3.0:1789760714.752506:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620096 00000100:00100000:3.0:1789760714.752559:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693122038827 time 693165249871 state 0x0 ref 1 00000100:00100000:3.0:1789760714.752574:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693165510287 time 693165249871 state 0x1 ref 1 00000100:00100000:3.0:1789760714.752612:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693165510287 time 693217900454 state 0x0 ref 1 00000100:00100000:3.0:1789760714.752625:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515620096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.752678:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19951616, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:3.0:1789760714.752687:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.752703:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760714.752730:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.752737:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.752750:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.785065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972188 is committed 00002000:00010000:3.0:1789760714.785248:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19951616 -> 19955712 00010000:00010000:3.0:1789760714.785264:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.785278:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.785358:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.785367:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.785373:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.785383:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.785403:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.785412:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d79b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.785546:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515620096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32918us (33122us total) trans 4294972188 rc 0/0 00000100:00100000:3.0:1789760714.785557:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4900 00000100:00100000:3.0:1789760714.785567:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693165510287 time 693217900454 state 0x2 ref 0 00000100:00100000:1.0:1789760714.793664:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.793797:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620224 00000100:00100000:3.0:1789760714.793808:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620224 00000100:00100000:3.0:1789760714.793881:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693165510287 time 693217900454 state 0x0 ref 1 00000100:00100000:3.0:1789760714.793899:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693218160870 time 693217900454 state 0x1 ref 1 00000100:00100000:3.0:1789760714.793933:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693218160870 time 693259222258 state 0x0 ref 1 00000100:00100000:3.0:1789760714.793947:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515620224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.794007:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19955712, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:3.0:1789760714.794018:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.794036:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760714.794042:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.794048:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.794064:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.842468:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972189 is committed 00002000:00010000:3.0:1789760714.842609:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19955712 -> 19959808 00010000:00010000:3.0:1789760714.842624:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.842636:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.842645:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.842652:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.842657:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.842666:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.842691:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.842735:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.843023:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515620224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49074us (49365us total) trans 4294972189 rc 0/0 00000100:00100000:3.0:1789760714.843034:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4901 00000100:00100000:3.0:1789760714.843043:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693218160870 time 693259222258 state 0x2 ref 0 00000100:00100000:2.0:1789760714.857965:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760714.858104:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620352 00000100:00100000:1.0:1789760714.858114:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620352 00000100:00100000:1.0:1789760714.858162:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693218160870 time 693259222258 state 0x0 ref 1 00000100:00100000:1.0:1789760714.858176:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693259482674 time 693259222258 state 0x1 ref 1 00000100:00100000:1.0:1789760714.858210:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693259482674 time 693323500095 state 0x0 ref 1 00000100:00100000:1.0:1789760714.858221:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515620352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760714.858268:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19959808, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa007 00010000:00010000:1.0:1789760714.858276:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.858291:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.858296:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.858302:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.858315:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.897801:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19959808 -> 19963904 00002000:00010000:1.0:1789760714.897810:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760714 -> 1789760715 00002000:00010000:1.0:1789760714.897813:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760714 -> 1789760715 00010000:00010000:1.0:1789760714.897833:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.897847:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.897857:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.897864:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760714.897869:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.897879:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.897899:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.897906:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.897997:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515620352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39771us (40036us total) trans 4294972190 rc 0/0 00000100:00100000:1.0:1789760714.898008:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4902 00000100:00100000:1.0:1789760714.898017:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693259482674 time 693323500095 state 0x2 ref 0 00000001:00080000:3.0:1789760714.900734:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972190 is committed 00000100:00100000:0.0:1789760714.902134:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.902226:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620480 00000100:00100000:2.0:1789760714.902235:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620480 00000100:00100000:2.0:1789760714.902304:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693259482674 time 693323500095 state 0x0 ref 1 00000100:00100000:2.0:1789760714.902322:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693323760511 time 693323500095 state 0x1 ref 1 00000100:00100000:2.0:1789760714.902356:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693323760511 time 693367644708 state 0x0 ref 1 00000100:00100000:2.0:1789760714.902368:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515620480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.902422:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19963904, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:2.0:1789760714.902431:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.902445:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.902449:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.902455:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.902474:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.924196:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972191 is committed 00002000:00010000:3.0:1789760714.924331:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19963904 -> 19968000 00010000:00010000:3.0:1789760714.924358:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.924375:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.924385:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.924393:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.924398:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.924412:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.924432:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.924440:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1d7b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.924538:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515620480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22167us (22406us total) trans 4294972191 rc 0/0 00000100:00100000:3.0:1789760714.924549:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4903 00000100:00100000:3.0:1789760714.924559:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693323760511 time 693367644708 state 0x2 ref 0 00000100:00100000:0.0:1789760714.930537:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760714.930596:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620608 00000100:00100000:2.0:1789760714.930606:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620608 00000100:00100000:2.0:1789760714.930679:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693323760511 time 693367644708 state 0x0 ref 1 00000100:00100000:2.0:1789760714.930696:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693367905124 time 693367644708 state 0x1 ref 1 00000100:00100000:2.0:1789760714.930749:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693367905124 time 693396037447 state 0x0 ref 1 00000100:00100000:2.0:1789760714.930763:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515620608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760714.930813:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19968000, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:2.0:1789760714.930822:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.930854:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.930859:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.930866:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.930880:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.970389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972192 is committed 00002000:00010000:3.0:1789760714.970681:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19968000 -> 19972096 00010000:00010000:3.0:1789760714.970695:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.970727:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.970737:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.970745:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760714.970750:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.970760:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.970779:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.970787:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1d7b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.970877:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515620608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40112us (40348us total) trans 4294972192 rc 0/0 00000100:00100000:3.0:1789760714.970887:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4904 00000100:00100000:3.0:1789760714.970897:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693367905124 time 693396037447 state 0x2 ref 0 00000100:00100000:2.0:1789760714.977081:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760714.977175:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620736 00000100:00100000:3.0:1789760714.977202:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620736 00000100:00100000:3.0:1789760714.977260:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693367905124 time 693396037447 state 0x0 ref 1 00000100:00100000:3.0:1789760714.977275:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693396297863 time 693396037447 state 0x1 ref 1 00000100:00100000:3.0:1789760714.977307:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693396297863 time 693442595611 state 0x0 ref 1 00000100:00100000:3.0:1789760714.977318:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515620736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760714.977365:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19972096, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760714.977374:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.977389:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760714.977393:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.977400:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.977413:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.007727:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972193 is committed 00002000:00010000:3.0:1789760715.007826:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19972096 -> 19976192 00010000:00010000:3.0:1789760715.007841:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.007853:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.007862:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.007869:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.007874:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.007882:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.007908:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.007978:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.008110:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515620736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30781us (31026us total) trans 4294972193 rc 0/0 00000100:00100000:3.0:1789760715.008121:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4905 00000100:00100000:3.0:1789760715.008130:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693396297863 time 693442595611 state 0x2 ref 0 00000100:00100000:0.0:1789760715.012809:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.012953:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620864 00000100:00100000:3.0:1789760715.012964:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620864 00000100:00100000:3.0:1789760715.013014:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693396297863 time 693442595611 state 0x0 ref 1 00000100:00100000:3.0:1789760715.013030:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693442856027 time 693442595611 state 0x1 ref 1 00000100:00100000:3.0:1789760715.013063:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693442856027 time 693478351677 state 0x0 ref 1 00000100:00100000:3.0:1789760715.013076:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515620864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.013130:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19976192, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.013140:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.013155:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.013160:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.013167:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.013328:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.054072:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972194 is committed 00002000:00010000:3.0:1789760715.054176:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19976192 -> 19980288 00010000:00010000:3.0:1789760715.054268:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.054281:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.054346:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.054354:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.054359:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.054367:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.054387:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.054396:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.054472:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515620864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41393us (41666us total) trans 4294972194 rc 0/0 00000100:00100000:3.0:1789760715.054481:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4906 00000100:00100000:3.0:1789760715.054491:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693442856027 time 693478351677 state 0x2 ref 0 00000100:00100000:0.0:1789760715.057826:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.057919:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515620992 00000100:00100000:3.0:1789760715.057929:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515620992 00000100:00100000:3.0:1789760715.057980:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693442856027 time 693478351677 state 0x0 ref 1 00000100:00100000:3.0:1789760715.057994:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693478612093 time 693478351677 state 0x1 ref 1 00000100:00100000:3.0:1789760715.058029:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693478612093 time 693523318766 state 0x0 ref 1 00000100:00100000:3.0:1789760715.058039:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515620992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.058092:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19980288, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.058101:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.058116:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.058121:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.058128:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.058146:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.070212:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760715.105635:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972195 is committed 00002000:00010000:3.0:1789760715.105781:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19980288 -> 19984384 00010000:00010000:3.0:1789760715.105796:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.105808:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.105818:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.105825:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.105830:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.105840:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.105859:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.105867:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.105968:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515620992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47924us (48147us total) trans 4294972195 rc 0/0 00000100:00100000:3.0:1789760715.105979:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4907 00000100:00100000:3.0:1789760715.105988:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693478612093 time 693523318766 state 0x2 ref 0 00000100:00100000:2.0:1789760715.112290:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760715.112393:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621120 00000100:00100000:1.0:1789760715.112403:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621120 00000100:00100000:1.0:1789760715.112462:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693478612093 time 693523318766 state 0x0 ref 1 00000100:00100000:1.0:1789760715.112476:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693523579182 time 693523318766 state 0x1 ref 1 00000100:00100000:1.0:1789760715.112502:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693523579182 time 693577791105 state 0x0 ref 1 00000100:00100000:1.0:1789760715.112514:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760715.112559:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19984384, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:1.0:1789760715.112568:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.112583:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760715.112587:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.112593:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.112608:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760715.132421:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19984384 -> 19988480 00000001:00080000:2.0:1789760715.132421:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972196 is committed 00010000:00010000:1.0:1789760715.132438:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.132451:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.132461:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.132468:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760715.132473:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.132482:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.132505:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.132511:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.132586:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 20070us (20302us total) trans 4294972196 rc 0/0 00000100:00100000:1.0:1789760715.132597:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4908 00000100:00100000:1.0:1789760715.132608:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693523579182 time 693577791105 state 0x2 ref 0 00000100:00100000:0.0:1789760715.140540:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760715.141625:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621248 00000100:00100000:2.0:1789760715.141637:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621248 00000100:00100000:2.0:1789760715.141695:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693523579182 time 693577791105 state 0x0 ref 1 00000100:00100000:2.0:1789760715.141745:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693578051521 time 693577791105 state 0x1 ref 1 00000100:00100000:3.0:1789760715.142121:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693578051521 time 693607406551 state 0x0 ref 1 00000100:00100000:3.0:1789760715.142137:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515621248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.142293:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19988480, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.142303:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.142318:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.142323:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.142329:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.142343:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.179329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972197 is committed 00002000:00010000:2.0:1789760715.179613:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19988480 -> 19992576 00010000:00010000:2.0:1789760715.179627:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.179640:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.179649:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.179656:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760715.179661:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.179670:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.179689:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.179697:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1d7da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.179862:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515621248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37723us (39332us total) trans 4294972197 rc 0/0 00000100:00100000:2.0:1789760715.179872:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4909 00000100:00100000:2.0:1789760715.179881:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693578051521 time 693607406551 state 0x2 ref 0 00000100:00100000:0.0:1789760715.186099:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760715.186313:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621376 00000100:00100000:2.0:1789760715.186323:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621376 00000100:00100000:2.0:1789760715.186377:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693578051521 time 693607406551 state 0x0 ref 1 00000100:00100000:2.0:1789760715.186392:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693607666967 time 693607406551 state 0x1 ref 1 00000100:00100000:2.0:1789760715.186421:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693607666967 time 693651710508 state 0x0 ref 1 00000100:00100000:2.0:1789760715.186435:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760715.186479:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19992576, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:2.0:1789760715.186489:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.186505:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760715.186511:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.186517:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.186531:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.219967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972198 is committed 00002000:00010000:2.0:1789760715.220072:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19992576 -> 19996672 00010000:00010000:2.0:1789760715.220088:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.220155:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.220166:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.220174:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760715.220187:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.220257:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.220277:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.220291:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.220418:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33980us (34323us total) trans 4294972198 rc 0/0 00000100:00100000:2.0:1789760715.220429:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4910 00000100:00100000:2.0:1789760715.220439:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693607666967 time 693651710508 state 0x2 ref 0 00000100:00100000:1.0:1789760715.225576:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.225652:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621504 00000100:00100000:3.0:1789760715.225662:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621504 00000100:00100000:3.0:1789760715.225730:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693607666967 time 693651710508 state 0x0 ref 1 00000100:00100000:3.0:1789760715.225745:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693651970924 time 693651710508 state 0x1 ref 1 00000100:00100000:3.0:1789760715.225818:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693651970924 time 693691106708 state 0x0 ref 1 00000100:00100000:3.0:1789760715.225831:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.225880:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 19996672, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.225889:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.225905:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.225920:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.225927:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.225942:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.253046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972199 is committed 00002000:00010000:3.0:1789760715.253131:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 19996672 -> 20000768 00010000:00010000:3.0:1789760715.253146:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.253158:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.253168:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.253196:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.253202:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.253211:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.253231:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.253240:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d7e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.253323:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27490us (27752us total) trans 4294972199 rc 0/0 00000100:00100000:3.0:1789760715.253333:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4911 00000100:00100000:3.0:1789760715.253342:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693651970924 time 693691106708 state 0x2 ref 0 00000100:00100000:1.0:1789760715.260349:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760715.260630:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621632 00000100:00100000:2.0:1789760715.260641:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621632 00000100:00100000:2.0:1789760715.260695:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693651970924 time 693691106708 state 0x0 ref 1 00000100:00100000:2.0:1789760715.260749:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693691367124 time 693691106708 state 0x1 ref 1 00000100:00100000:2.0:1789760715.260780:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693691367124 time 693726069372 state 0x0 ref 1 00000100:00100000:2.0:1789760715.260793:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515621632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760715.260940:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20000768, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:2.0:1789760715.260949:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.260964:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760715.260968:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.260974:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.260992:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.295633:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20000768 -> 20004864 00010000:00010000:2.0:1789760715.295655:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.295668:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.295677:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.295685:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760715.295691:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.295699:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.295737:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.295745:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1d7ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.295852:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515621632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35056us (35507us total) trans 4294972200 rc 0/0 00000100:00100000:2.0:1789760715.295863:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4912 00000100:00100000:2.0:1789760715.295872:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693691367124 time 693726069372 state 0x2 ref 0 00000001:00080000:0.0:1789760715.297793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972200 is committed 00000100:00100000:3.0:1789760715.300083:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760715.300477:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621760 00000100:00100000:0.0:1789760715.300488:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621760 00000100:00100000:0.0:1789760715.300539:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693691367124 time 693726069372 state 0x0 ref 1 00000100:00100000:0.0:1789760715.300555:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693726329788 time 693726069372 state 0x1 ref 1 00000100:00100000:0.0:1789760715.300587:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693726329788 time 693765876827 state 0x0 ref 1 00000100:00100000:0.0:1789760715.300600:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515621760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760715.300645:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20004864, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:0.0:1789760715.300655:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.300671:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760715.300676:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.300682:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.300696:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.332604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972201 is committed 00002000:00010000:1.0:1789760715.332775:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20004864 -> 20008960 00010000:00010000:1.0:1789760715.332792:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.332805:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.332814:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.332885:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760715.332891:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.332901:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.332923:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.332931:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d7f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.333026:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515621760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32423us (32947us total) trans 4294972201 rc 0/0 00000100:00100000:1.0:1789760715.333036:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4913 00000100:00100000:1.0:1789760715.333045:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693726329788 time 693765876827 state 0x2 ref 0 00000100:00100000:2.0:1789760715.340053:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760715.340159:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515621888 00000100:00100000:1.0:1789760715.340170:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515621888 00000100:00100000:1.0:1789760715.340250:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693726329788 time 693765876827 state 0x0 ref 1 00000100:00100000:1.0:1789760715.340268:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693766137243 time 693765876827 state 0x1 ref 1 00000100:00100000:1.0:1789760715.340299:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693766137243 time 693805587753 state 0x0 ref 1 00000100:00100000:1.0:1789760715.340313:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515621888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760715.340364:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20008960, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:1.0:1789760715.340373:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.340391:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760715.340396:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.340403:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.340419:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760715.372845:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20008960 -> 20013056 00010000:00010000:3.0:1789760715.372865:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.372878:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.372895:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.372903:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.372908:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.372917:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.372936:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.372944:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1d7fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.373034:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515621888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32719us (32986us total) trans 4294972202 rc 0/0 00000100:00100000:3.0:1789760715.373045:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4914 00000100:00100000:3.0:1789760715.373055:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693766137243 time 693805587753 state 0x2 ref 0 00000001:00080000:0.0:1789760715.373211:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972202 is committed 00000100:00100000:2.0:1789760715.376353:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.376846:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622016 00000100:00100000:3.0:1789760715.376857:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622016 00000100:00100000:3.0:1789760715.376908:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693766137243 time 693805587753 state 0x0 ref 1 00000100:00100000:3.0:1789760715.376929:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693805848169 time 693805587753 state 0x1 ref 1 00000100:00100000:3.0:1789760715.376959:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693805848169 time 693842248708 state 0x0 ref 1 00000100:00100000:3.0:1789760715.376972:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515622016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.377017:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20013056, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.377026:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.377040:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.377044:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.377051:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.377065:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.411754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972203 is committed 00002000:00010000:3.0:1789760715.412016:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20013056 -> 20017152 00010000:00010000:3.0:1789760715.412032:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.412047:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.412057:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.412064:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.412070:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.412079:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.412108:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.412117:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d804 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.412309:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515622016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35333us (35959us total) trans 4294972203 rc 0/0 00000100:00100000:3.0:1789760715.412322:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4915 00000100:00100000:3.0:1789760715.412332:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693805848169 time 693842248708 state 0x2 ref 0 00000100:00100000:2.0:1789760715.415782:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.415915:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622144 00000100:00100000:3.0:1789760715.415924:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622144 00000100:00100000:3.0:1789760715.415973:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693805848169 time 693842248708 state 0x0 ref 1 00000100:00100000:3.0:1789760715.415989:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693842509124 time 693842248708 state 0x1 ref 1 00000100:00100000:3.0:1789760715.416021:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693842509124 time 693881309835 state 0x0 ref 1 00000100:00100000:3.0:1789760715.416032:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515622144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.416075:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20017152, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.416084:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.416099:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.416103:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.416169:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.416251:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.444292:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972204 is committed 00002000:00010000:3.0:1789760715.444413:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20017152 -> 20021248 00010000:00010000:3.0:1789760715.444428:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.444441:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.444449:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.444456:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.444461:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.444469:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.444493:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.444501:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d80b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.444654:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515622144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28618us (28876us total) trans 4294972204 rc 0/0 00000100:00100000:3.0:1789760715.444665:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4916 00000100:00100000:3.0:1789760715.444675:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693842509124 time 693881309835 state 0x2 ref 0 00000100:00100000:2.0:1789760715.449409:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760715.449971:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622272 00000100:00100000:0.0:1789760715.449982:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622272 00000100:00100000:0.0:1789760715.450044:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693842509124 time 693881309835 state 0x0 ref 1 00000100:00100000:0.0:1789760715.450060:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693881570251 time 693881309835 state 0x1 ref 1 00000100:00100000:0.0:1789760715.450089:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693881570251 time 693915378894 state 0x0 ref 1 00000100:00100000:0.0:1789760715.450108:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9093daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515622272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760715.450154:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20021248, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:0.0:1789760715.450163:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.450388:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760715.450395:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.450402:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.450417:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.482505:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972205 is committed 00002000:00010000:1.0:1789760715.482667:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20021248 -> 20025344 00010000:00010000:1.0:1789760715.482683:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.482695:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.482704:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.482742:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760715.482748:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.482757:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.482783:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.482791:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d812 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.482883:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9093daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515622272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32772us (33478us total) trans 4294972205 rc 0/0 00000100:00100000:1.0:1789760715.482893:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4917 00000100:00100000:1.0:1789760715.482901:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693881570251 time 693915378894 state 0x2 ref 0 00000100:00100000:2.0:1789760715.487083:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.487271:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622400 00000100:00100000:3.0:1789760715.487281:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622400 00000100:00100000:3.0:1789760715.487333:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693881570251 time 693915378894 state 0x0 ref 1 00000100:00100000:3.0:1789760715.487347:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693915639310 time 693915378894 state 0x1 ref 1 00000100:00100000:3.0:1789760715.487378:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693915639310 time 693952666759 state 0x0 ref 1 00000100:00100000:3.0:1789760715.487390:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9093da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515622400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.487436:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20025344, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.487445:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.487461:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760715.487466:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.487472:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.487486:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.523072:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972206 is committed 00002000:00010000:3.0:1789760715.523265:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20025344 -> 20029440 00010000:00010000:3.0:1789760715.523280:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.523292:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.523302:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.523309:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.523315:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.523325:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.523344:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.523351:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d819 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.523440:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9093da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515622400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36047us (36362us total) trans 4294972206 rc 0/0 00000100:00100000:3.0:1789760715.523451:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4918 00000100:00100000:3.0:1789760715.523461:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693915639310 time 693952666759 state 0x2 ref 0 00000100:00100000:2.0:1789760715.527172:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760715.527407:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622528 00000100:00100000:1.0:1789760715.527418:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622528 00000100:00100000:1.0:1789760715.527469:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693915639310 time 693952666759 state 0x0 ref 1 00000100:00100000:1.0:1789760715.527485:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693952927175 time 693952666759 state 0x1 ref 1 00000100:00100000:1.0:1789760715.527512:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693952927175 time 693992801036 state 0x0 ref 1 00000100:00100000:1.0:1789760715.527524:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515622528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760715.527569:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20029440, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:1.0:1789760715.527579:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.527595:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760715.527602:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.527608:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.527622:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.556336:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972207 is committed 00002000:00010000:3.0:1789760715.556438:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20029440 -> 20033536 00010000:00010000:3.0:1789760715.556453:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.556467:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.556477:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.556485:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760715.556490:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.556498:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.556519:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.556533:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d820 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760715.556637:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515622528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29109us (29470us total) trans 4294972207 rc 0/0 00000100:00100000:3.0:1789760715.556647:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4919 00000100:00100000:3.0:1789760715.556657:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693952927175 time 693992801036 state 0x2 ref 0 00000100:00100000:2.0:1789760715.560834:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.561088:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622656 00000100:00100000:3.0:1789760715.561107:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622656 00000100:00100000:3.0:1789760715.561156:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693952927175 time 693992801036 state 0x0 ref 1 00000100:00100000:3.0:1789760715.561171:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693993061452 time 693992801036 state 0x1 ref 1 00000100:00100000:3.0:1789760715.561213:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693993061452 time 694026501512 state 0x0 ref 1 00000100:00100000:3.0:1789760715.561226:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515622656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.561272:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20033536, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.561281:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.561298:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760715.561303:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.561310:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.561329:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.594581:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972208 is committed 00002000:00010000:1.0:1789760715.594677:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20033536 -> 20037632 00010000:00010000:1.0:1789760715.594692:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.594725:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.594736:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.594743:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760715.594748:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.594757:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.594777:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.594786:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1d827 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.594952:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515622656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33724us (34132us total) trans 4294972208 rc 0/0 00000100:00100000:1.0:1789760715.594963:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4920 00000100:00100000:1.0:1789760715.594974:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693993061452 time 694026501512 state 0x2 ref 0 00000100:00100000:1.0:1789760715.598332:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760715.598508:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622784 00000100:00100000:2.0:1789760715.598519:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622784 00000100:00100000:2.0:1789760715.598575:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 693993061452 time 694026501512 state 0x0 ref 1 00000100:00100000:2.0:1789760715.598592:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694026761928 time 694026501512 state 0x1 ref 1 00000100:00100000:2.0:1789760715.598624:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694026761928 time 694063912798 state 0x0 ref 1 00000100:00100000:2.0:1789760715.598638:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515622784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760715.598689:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20037632, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:2.0:1789760715.598698:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.598936:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.598941:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.598947:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.598960:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.635076:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20037632 -> 20041728 00010000:00010000:2.0:1789760715.635105:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.635119:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.635128:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.635135:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760715.635139:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.635147:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.635167:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.635184:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d82e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.635391:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515622784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36750us (37062us total) trans 4294972209 rc 0/0 00000100:00100000:2.0:1789760715.635404:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4921 00000100:00100000:2.0:1789760715.635414:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694026761928 time 694063912798 state 0x2 ref 0 00000001:00080000:2.0:1789760715.636314:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972209 is committed 00000100:00100000:2.0:1789760715.645014:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760715.645148:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515622912 00000100:00100000:3.0:1789760715.645157:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515622912 00000100:00100000:3.0:1789760715.645225:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694026761928 time 694063912798 state 0x0 ref 1 00000100:00100000:3.0:1789760715.645240:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694064173214 time 694063912798 state 0x1 ref 1 00000100:00100000:3.0:1789760715.645273:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694064173214 time 694110562362 state 0x0 ref 1 00000100:00100000:3.0:1789760715.645286:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515622912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760715.645330:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20041728, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:3.0:1789760715.645339:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.645355:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760715.645359:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.645365:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.645377:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.729108:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972210 is committed 00002000:00010000:0.0:1789760715.730921:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20041728 -> 20045824 00010000:00010000:0.0:1789760715.730937:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.730951:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.730960:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.730968:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760715.730972:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.730982:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731001:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.731009:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1d835 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.731112:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515622912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 85824us (86103us total) trans 4294972210 rc 0/0 00000100:00100000:0.0:1789760715.731122:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4922 00000100:00100000:0.0:1789760715.731132:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694064173214 time 694110562362 state 0x2 ref 0 00000100:00100000:3.0:1789760715.741878:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760715.742034:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623040 00000100:00100000:1.0:1789760715.742045:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623040 00000100:00100000:1.0:1789760715.742098:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694064173214 time 694110562362 state 0x0 ref 1 00000100:00100000:1.0:1789760715.742113:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694110822778 time 694110562362 state 0x1 ref 1 00000100:00100000:1.0:1789760715.742143:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694110822778 time 694207432306 state 0x0 ref 1 00000100:00100000:1.0:1789760715.742155:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515623040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760715.742311:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20045824, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:1.0:1789760715.742320:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.742336:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760715.742341:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.742347:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.742361:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.788549:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972211 is committed 00002000:00010000:1.0:1789760715.788659:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20045824 -> 20049920 00010000:00010000:1.0:1789760715.788676:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.788688:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.788697:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.788704:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760715.788771:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.788780:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.788800:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.788808:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d83c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.788991:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515623040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46833us (47116us total) trans 4294972211 rc 0/0 00000100:00100000:1.0:1789760715.789006:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4923 00000100:00100000:1.0:1789760715.789015:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694110822778 time 694207432306 state 0x2 ref 0 00000100:00100000:1.0:1789760715.799658:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760715.799907:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623168 00000100:00100000:2.0:1789760715.799917:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623168 00000100:00100000:2.0:1789760715.799974:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694110822778 time 694207432306 state 0x0 ref 1 00000100:00100000:2.0:1789760715.799989:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694207692722 time 694207432306 state 0x1 ref 1 00000100:00100000:2.0:1789760715.800021:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694207692722 time 694265310744 state 0x0 ref 1 00000100:00100000:2.0:1789760715.800034:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515623168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760715.800081:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20049920, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:2.0:1789760715.800090:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.800115:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.800119:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.800125:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.800139:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.845655:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20049920 -> 20054016 00010000:00010000:2.0:1789760715.845677:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.845691:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.845701:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.845735:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760715.845741:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.845751:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.845771:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.845779:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d843 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.845933:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515623168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45896us (46279us total) trans 4294972212 rc 0/0 00000100:00100000:2.0:1789760715.845944:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4924 00000100:00100000:2.0:1789760715.845954:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694207692722 time 694265310744 state 0x2 ref 0 00000001:00080000:0.0:1789760715.849550:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972212 is committed 00000100:00100000:0.0:1789760715.850296:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760715.850384:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623296 00000100:00100000:1.0:1789760715.850394:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623296 00000100:00100000:0.0:1789760715.859548:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694207692722 time 694265310744 state 0x0 ref 1 00000100:00100000:0.0:1789760715.859567:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694265571160 time 694265310744 state 0x1 ref 1 00000100:00100000:0.0:1789760715.859607:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694265571160 time 694324896486 state 0x0 ref 1 00000100:00100000:0.0:1789760715.859622:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515623296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760715.859674:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20054016, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa007 00010000:00010000:0.0:1789760715.859683:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.859700:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.859736:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.859743:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.859758:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.937549:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20054016 -> 20058112 00002000:00010000:0.0:1789760715.937558:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760715 -> 1789760716 00002000:00010000:0.0:1789760715.937561:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760715 -> 1789760716 00010000:00010000:0.0:1789760715.937574:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.937587:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.937598:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.937604:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760715.937609:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.937618:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.937637:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.937645:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d84a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.937757:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515623296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 78133us (87459us total) trans 4294972213 rc 0/0 00000100:00100000:0.0:1789760715.937768:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4925 00000100:00100000:0.0:1789760715.937778:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694265571160 time 694324896486 state 0x2 ref 0 00000001:00080000:0.0:1789760715.937819:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972213 is committed 00000100:00100000:3.0:1789760715.947569:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760715.952284:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623424 00000100:00100000:1.0:1789760715.952301:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623424 00000100:00100000:1.0:1789760715.952368:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694265571160 time 694324896486 state 0x0 ref 1 00000100:00100000:1.0:1789760715.952384:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694325156902 time 694324896486 state 0x1 ref 1 00000100:00100000:1.0:1789760715.952823:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694325156902 time 694418098695 state 0x0 ref 1 00000100:00100000:1.0:1789760715.952841:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515623424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760715.952894:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20058112, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:1.0:1789760715.952903:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.952921:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760715.952926:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.952932:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.952962:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.001929:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972214 is committed 00002000:00010000:2.0:1789760716.002027:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20058112 -> 20062208 00010000:00010000:2.0:1789760716.002043:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.002056:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.002066:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.002074:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.002079:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.002087:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.002115:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.002129:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d851 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.002272:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515623424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49432us (54708us total) trans 4294972214 rc 0/0 00000100:00100000:2.0:1789760716.002284:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4926 00000100:00100000:2.0:1789760716.002293:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694325156902 time 694418098695 state 0x2 ref 0 00000100:00100000:0.0:1789760716.008159:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.008983:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623552 00000100:00100000:2.0:1789760716.008993:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623552 00000100:00100000:2.0:1789760716.009067:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694325156902 time 694418098695 state 0x0 ref 1 00000100:00100000:2.0:1789760716.009084:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694418359111 time 694418098695 state 0x1 ref 1 00000100:00100000:2.0:1789760716.009124:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694418359111 time 694474412496 state 0x0 ref 1 00000100:00100000:2.0:1789760716.009137:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cad050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515623552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.009195:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20062208, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:2.0:1789760716.009204:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.009219:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760716.009224:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.009231:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.009251:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760716.009734:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760716.054724:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972215 is committed 00002000:00010000:0.0:1789760716.054943:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20062208 -> 20066304 00010000:00010000:0.0:1789760716.054959:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.054973:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.054981:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.054989:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760716.054995:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.055004:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.055024:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.055031:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d858 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.055123:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cad050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515623552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45984us (47027us total) trans 4294972215 rc 0/0 00000100:00100000:0.0:1789760716.055133:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4927 00000100:00100000:0.0:1789760716.055142:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694418359111 time 694474412496 state 0x2 ref 0 00000100:00100000:0.0:1789760716.061996:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760716.062088:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623680 00000100:00100000:1.0:1789760716.062099:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623680 00000100:00100000:1.0:1789760716.062153:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694418359111 time 694474412496 state 0x0 ref 1 00000100:00100000:1.0:1789760716.062168:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694474672912 time 694474412496 state 0x1 ref 1 00000100:00100000:1.0:1789760716.062208:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694474672912 time 694527497306 state 0x0 ref 1 00000100:00100000:1.0:1789760716.062221:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515623680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760716.062267:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20066304, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:1.0:1789760716.062276:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.062291:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760716.062295:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.062301:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.062315:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.093015:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972216 is committed 00002000:00010000:0.0:1789760716.093415:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20066304 -> 20070400 00010000:00010000:0.0:1789760716.093430:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.093446:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.093455:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.093463:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760716.093467:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.093475:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.093494:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.093502:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d85f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.093662:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515623680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31437us (31672us total) trans 4294972216 rc 0/0 00000100:00100000:0.0:1789760716.093672:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4928 00000100:00100000:0.0:1789760716.093682:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694474672912 time 694527497306 state 0x2 ref 0 00010000:00010000:3.0:1789760716.094696:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760716.098145:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760716.098207:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623808 00000100:00100000:3.0:1789760716.098217:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623808 00000100:00100000:3.0:1789760716.098273:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694474672912 time 694527497306 state 0x0 ref 1 00000100:00100000:3.0:1789760716.098288:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694527757722 time 694527497306 state 0x1 ref 1 00000100:00100000:3.0:1789760716.098319:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694527757722 time 694563608349 state 0x0 ref 1 00000100:00100000:3.0:1789760716.098334:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515623808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760716.098381:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20070400, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:3.0:1789760716.098390:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.098407:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760716.098411:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.098418:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.098431:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760716.155977:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20070400 -> 20074496 00010000:00010000:3.0:1789760716.155999:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.156018:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.156027:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.156035:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760716.156040:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.156050:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.156074:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.156370:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d866 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.156475:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515623808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58131us (58326us total) trans 4294972217 rc 0/0 00000100:00100000:3.0:1789760716.156486:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4929 00000100:00100000:3.0:1789760716.156497:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694527757722 time 694563608349 state 0x2 ref 0 00000001:00080000:2.0:1789760716.156586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972217 is committed 00000100:00100000:1.0:1789760716.160008:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760716.161306:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515623936 00000100:00100000:3.0:1789760716.161321:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515623936 00000100:00100000:3.0:1789760716.161378:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694527757722 time 694563608349 state 0x0 ref 1 00000100:00100000:3.0:1789760716.161395:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694563868765 time 694563608349 state 0x1 ref 1 00000100:00100000:3.0:1789760716.161474:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694563868765 time 694626753382 state 0x0 ref 1 00000100:00100000:3.0:1789760716.161488:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515623936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760716.161545:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20074496, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:3.0:1789760716.161554:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.161570:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760716.161576:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.161582:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.161597:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.247492:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972218 is committed 00002000:00010000:3.0:1789760716.247646:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20074496 -> 20078592 00010000:00010000:3.0:1789760716.247662:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.247675:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.247688:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.247694:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760716.247699:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.247724:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.247745:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.247752:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d86d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.247877:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515623936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 86386us (87874us total) trans 4294972218 rc 0/0 00000100:00100000:3.0:1789760716.247887:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4930 00000100:00100000:3.0:1789760716.247896:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694563868765 time 694626753382 state 0x2 ref 0 00000100:00100000:0.0:1789760716.253687:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.253789:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624064 00000100:00100000:2.0:1789760716.253800:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624064 00000100:00100000:2.0:1789760716.253902:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694563868765 time 694626753382 state 0x0 ref 1 00000100:00100000:2.0:1789760716.253914:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694627013798 time 694626753382 state 0x1 ref 1 00000100:00100000:0.0:1789760716.253974:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694627013798 time 694719261043 state 0x0 ref 1 00000100:00100000:0.0:1789760716.253988:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515624064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760716.254038:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20078592, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:0.0:1789760716.254047:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.254064:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760716.254068:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.254074:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.254088:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.285858:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972219 is committed 00000100:00100000:2.0:1789760716.286049:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae802f82d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544407168:0@lo:400:kworker.0 00000100:00100000:1.0:1789760716.286286:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae802f83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544406784:0@lo:400:kworker.0 00002000:00010000:3.0:1789760716.286406:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20078592 -> 20082688 00010000:00010000:3.0:1789760716.286421:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286438:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286447:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286455:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760716.286460:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286469:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286488:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.286496:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d874 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.286565:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760716.286612:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93bfd9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544407296:0@lo:400:kworker.0 00000100:00100000:1.0:1789760716.286638:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760716.286799:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760716.286868:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae802f81180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544406912:0@lo:400:kworker.0 00000100:00100000:2.0:1789760716.286895:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760716.286921:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93bfd9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544407424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760716.286956:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760716.286998:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407296 00000100:00100000:1.0:1789760716.287005:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407296 00000100:00100000:1.0:1789760716.287053:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406912 00000100:00100000:1.0:1789760716.287056:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406912 00000100:00100000:1.0:1789760716.287076:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1789760716.287085:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544407296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760716.287292:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515624064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33303us (33610us total) trans 4294972219 rc 0/0 00000100:00100000:3.0:1789760716.287302:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4931 00000100:00100000:3.0:1789760716.287312:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694627013798 time 694719261043 state 0x2 ref 0 00000100:00100000:3.0:1789760716.287343:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544406784 00000100:00100000:3.0:1789760716.287347:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544406784 00000100:00100000:3.0:1789760716.287374:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1789760716.287382:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9092df850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544406784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760716.287429:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9092df850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544406784:12345-0@lo:400:kworker.0 Request processed in 48us (864us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760716.287436:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1789760716.287465:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1789760716.287473:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544406912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760716.287502:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544406912:12345-0@lo:400:kworker.0 Request processed in 30us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760716.287509:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1789760716.287639:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae802f83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544406784:0@lo:400:kworker.0 00000100:00100000:1.0:1789760716.287964:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544407296:12345-0@lo:400:kworker.0 Request processed in 880us (1324us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760716.287971:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00080000:1.0:1789760716.287988:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760716.288118:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93bfd9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544407296:0@lo:400:kworker.0 00000100:00100000:0.0:1789760716.288218:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407168 00000100:00100000:0.0:1789760716.288224:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407168 00000100:00100000:1.0:1789760716.288241:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407424 00000100:00100000:1.0:1789760716.288248:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407424 00000100:00100000:2.0:1789760716.288250:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae802f81180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544406912:0@lo:400:kworker.0 00000100:00100000:2.0:1789760716.288265:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760716.288272:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae802f81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544407040:0@lo:400:kworker.0 00000100:00100000:1.0:1789760716.288295:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:1.0:1789760716.288307:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae918867c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544407424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760716.288307:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760716.288337:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:0.0:1789760716.288348:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544407168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760716.288363:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae918867c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544407424:12345-0@lo:400:kworker.0 Request processed in 59us (1404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760716.288364:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407040 00000100:00100000:2.0:1789760716.288370:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407040 00000100:00100000:1.0:1789760716.288374:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1789760716.288405:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544407168:12345-0@lo:400:kworker.0 Request processed in 59us (1605us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760716.288413:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93bfd9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544407424:0@lo:400:kworker.0 00000100:00100000:0.0:1789760716.288417:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789760716.288472:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00080000:3.0:1789760716.288481:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760716.288483:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544407040:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789760716.288488:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760716.288493:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760716.288536:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544407040:12345-0@lo:400:kworker.0 Request processed in 56us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760716.288544:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1789760716.288704:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae802f81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544407040:0@lo:400:kworker.0 00000100:00100000:0.0:1789760716.288766:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae802f82d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544407168:0@lo:400:kworker.0 00000100:00100000:2.0:1789760716.299786:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760716.299934:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624192 00000100:00100000:1.0:1789760716.299943:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624192 00000100:00100000:1.0:1789760716.299996:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694627013798 time 694719261043 state 0x0 ref 1 00000100:00100000:1.0:1789760716.300011:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694719521459 time 694719261043 state 0x1 ref 1 00000100:00100000:1.0:1789760716.300044:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694719521459 time 694765333983 state 0x0 ref 1 00000100:00100000:1.0:1789760716.300057:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515624192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760716.300227:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20082688, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:1.0:1789760716.300237:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.300258:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760716.300263:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.300270:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.300284:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.331362:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972220 is committed 00002000:00010000:2.0:1789760716.331476:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20082688 -> 20086784 00010000:00010000:2.0:1789760716.331493:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.331506:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.331515:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.331666:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.331672:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.331683:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.331736:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.331751:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1d87b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.331838:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515624192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31777us (32058us total) trans 4294972220 rc 0/0 00000100:00100000:2.0:1789760716.331847:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4932 00000100:00100000:2.0:1789760716.331857:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694719521459 time 694765333983 state 0x2 ref 0 00000100:00100000:3.0:1789760716.338652:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.338753:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624320 00000100:00100000:0.0:1789760716.338765:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624320 00000100:00100000:0.0:1789760716.338827:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694719521459 time 694765333983 state 0x0 ref 1 00000100:00100000:0.0:1789760716.338843:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694765594399 time 694765333983 state 0x1 ref 1 00000100:00100000:0.0:1789760716.338871:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694765594399 time 694804160232 state 0x0 ref 1 00000100:00100000:0.0:1789760716.338883:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515624320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760716.338933:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20086784, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:0.0:1789760716.338941:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.338958:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760716.338961:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.338968:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.338981:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.374568:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20086784 -> 20090880 00010000:00010000:2.0:1789760716.374586:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.374598:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.374606:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.374612:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.374616:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.374625:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.374649:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.374656:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d882 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.374780:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515624320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35894us (36134us total) trans 4294972221 rc 0/0 00000100:00100000:2.0:1789760716.374789:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4933 00000100:00100000:2.0:1789760716.374799:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694765594399 time 694804160232 state 0x2 ref 0 00000001:00080000:1.0:1789760716.375812:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972221 is committed 00000100:00100000:0.0:1789760716.378147:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.378290:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624448 00000100:00100000:2.0:1789760716.378299:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624448 00000100:00100000:2.0:1789760716.378350:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694765594399 time 694804160232 state 0x0 ref 1 00000100:00100000:2.0:1789760716.378364:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694804420648 time 694804160232 state 0x1 ref 1 00000100:00100000:2.0:1789760716.378391:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694804420648 time 694843680866 state 0x0 ref 1 00000100:00100000:2.0:1789760716.378403:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.378448:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20090880, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:2.0:1789760716.378456:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.378472:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760716.378477:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.378483:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.378497:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.409484:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972222 is committed 00002000:00010000:2.0:1789760716.409576:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20090880 -> 20094976 00010000:00010000:2.0:1789760716.409590:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.409603:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.409613:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.409619:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.409624:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.409632:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.409652:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.409664:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d889 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.409843:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31436us (31700us total) trans 4294972222 rc 0/0 00000100:00100000:2.0:1789760716.409853:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4934 00000100:00100000:2.0:1789760716.409862:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694804420648 time 694843680866 state 0x2 ref 0 00000100:00100000:3.0:1789760716.413535:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.413655:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624576 00000100:00100000:0.0:1789760716.413664:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624576 00000100:00100000:0.0:1789760716.413747:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694804420648 time 694843680866 state 0x0 ref 1 00000100:00100000:0.0:1789760716.413762:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694843941282 time 694843680866 state 0x1 ref 1 00000100:00100000:0.0:1789760716.413786:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694843941282 time 694879075422 state 0x0 ref 1 00000100:00100000:0.0:1789760716.413798:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515624576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760716.413857:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20094976, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:0.0:1789760716.413866:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.413881:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760716.413886:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.413892:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.413905:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.439097:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972223 is committed 00002000:00010000:1.0:1789760716.439187:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20094976 -> 20099072 00010000:00010000:1.0:1789760716.439203:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.439216:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.439225:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.439238:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760716.439243:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.439253:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.439272:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.439280:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d890 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.439368:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515624576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25567us (25838us total) trans 4294972223 rc 0/0 00000100:00100000:1.0:1789760716.439379:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4935 00000100:00100000:1.0:1789760716.439390:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694843941282 time 694879075422 state 0x2 ref 0 00000100:00100000:1.0:1789760716.442340:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760716.442413:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624704 00000100:00100000:3.0:1789760716.442422:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624704 00000100:00100000:3.0:1789760716.442472:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694843941282 time 694879075422 state 0x0 ref 1 00000100:00100000:3.0:1789760716.442487:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694879335838 time 694879075422 state 0x1 ref 1 00000100:00100000:3.0:1789760716.442523:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694879335838 time 694907811932 state 0x0 ref 1 00000100:00100000:3.0:1789760716.442536:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760716.442584:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20099072, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:3.0:1789760716.442594:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.442609:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760716.442614:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.442620:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.442633:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.479569:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972224 is committed 00002000:00010000:2.0:1789760716.479771:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20099072 -> 20103168 00010000:00010000:2.0:1789760716.479786:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.479798:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.479808:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.479885:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.479891:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.479901:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.479926:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.479934:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1d897 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.480026:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37487us (37689us total) trans 4294972224 rc 0/0 00000100:00100000:2.0:1789760716.480036:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4936 00000100:00100000:2.0:1789760716.480046:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694879335838 time 694907811932 state 0x2 ref 0 00000100:00100000:0.0:1789760716.485776:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760716.485855:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624832 00000100:00100000:3.0:1789760716.485865:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624832 00000100:00100000:3.0:1789760716.485918:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694879335838 time 694907811932 state 0x0 ref 1 00000100:00100000:3.0:1789760716.485933:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694908072348 time 694907811932 state 0x1 ref 1 00000100:00100000:3.0:1789760716.485959:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694908072348 time 694951248433 state 0x0 ref 1 00000100:00100000:3.0:1789760716.485972:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760716.486019:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20103168, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:3.0:1789760716.486028:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.486044:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760716.486048:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.486054:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.486068:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.517604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972225 is committed 00002000:00010000:0.0:1789760716.517704:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20103168 -> 20107264 00010000:00010000:0.0:1789760716.517738:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.517757:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.517767:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.517775:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760716.517781:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.517790:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.517822:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.517831:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d89e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.517918:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31943us (32147us total) trans 4294972225 rc 0/0 00000100:00100000:0.0:1789760716.517928:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4937 00000100:00100000:0.0:1789760716.517938:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694908072348 time 694951248433 state 0x2 ref 0 00000100:00100000:3.0:1789760716.521324:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.521584:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515624960 00000100:00100000:2.0:1789760716.521593:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515624960 00000100:00100000:2.0:1789760716.521642:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694908072348 time 694951248433 state 0x0 ref 1 00000100:00100000:2.0:1789760716.521656:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694951508849 time 694951248433 state 0x1 ref 1 00000100:00100000:2.0:1789760716.521686:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694951508849 time 694986975148 state 0x0 ref 1 00000100:00100000:2.0:1789760716.521698:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.521779:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20107264, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:2.0:1789760716.521789:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.521805:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760716.521810:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.521822:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.521836:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.555590:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972226 is committed 00002000:00010000:1.0:1789760716.555694:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20107264 -> 20111360 00010000:00010000:1.0:1789760716.555742:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.555755:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.555766:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.555775:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760716.555780:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.555790:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.555822:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.555831:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1d8a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.555915:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515624960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34213us (34594us total) trans 4294972226 rc 0/0 00000100:00100000:1.0:1789760716.555925:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4938 00000100:00100000:1.0:1789760716.555934:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694951508849 time 694986975148 state 0x2 ref 0 00000100:00100000:1.0:1789760716.559632:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.559763:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625088 00000100:00100000:2.0:1789760716.559773:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625088 00000100:00100000:2.0:1789760716.559896:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694951508849 time 694986975148 state 0x0 ref 1 00000100:00100000:2.0:1789760716.559912:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694987235564 time 694986975148 state 0x1 ref 1 00000100:00100000:2.0:1789760716.559991:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694987235564 time 695025278263 state 0x0 ref 1 00000100:00100000:2.0:1789760716.560005:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.560051:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20111360, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:2.0:1789760716.560061:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.560077:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760716.560082:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.560101:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.560117:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.609558:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972227 is committed 00002000:00010000:2.0:1789760716.609660:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20111360 -> 20115456 00010000:00010000:2.0:1789760716.609676:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.609688:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.609697:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.609722:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.609728:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.609737:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.609757:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.609764:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.609958:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49951us (50331us total) trans 4294972227 rc 0/0 00000100:00100000:2.0:1789760716.609973:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4939 00000100:00100000:2.0:1789760716.609983:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694987235564 time 695025278263 state 0x2 ref 0 00000100:00100000:0.0:1789760716.613657:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.613760:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625216 00000100:00100000:2.0:1789760716.613771:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625216 00000100:00100000:2.0:1789760716.613839:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 694987235564 time 695025278263 state 0x0 ref 1 00000100:00100000:2.0:1789760716.613857:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695025538679 time 695025278263 state 0x1 ref 1 00000100:00100000:2.0:1789760716.613892:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695025538679 time 695079180554 state 0x0 ref 1 00000100:00100000:2.0:1789760716.613906:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.613953:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20115456, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:2.0:1789760716.613962:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.613979:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760716.613984:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.613992:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.614007:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.672299:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972228 is committed 00002000:00010000:2.0:1789760716.672328:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20115456 -> 20119552 00010000:00010000:2.0:1789760716.672344:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.672358:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.672368:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.672375:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.672380:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.672388:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.672405:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.672410:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1d8b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.672489:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58583us (58838us total) trans 4294972228 rc 0/0 00000100:00100000:2.0:1789760716.672499:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4940 00000100:00100000:2.0:1789760716.672509:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695025538679 time 695079180554 state 0x2 ref 0 00000100:00100000:0.0:1789760716.679949:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760716.680090:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625344 00000100:00100000:1.0:1789760716.680101:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625344 00000100:00100000:1.0:1789760716.680154:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695025538679 time 695079180554 state 0x0 ref 1 00000100:00100000:1.0:1789760716.680185:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695079440970 time 695079180554 state 0x1 ref 1 00000100:00100000:1.0:1789760716.680217:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695079440970 time 695145505754 state 0x0 ref 1 00000100:00100000:1.0:1789760716.680229:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760716.680277:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20119552, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:1.0:1789760716.680286:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.680301:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.680306:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.680312:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.680326:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.712634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972229 is committed 00002000:00010000:0.0:1789760716.712797:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20119552 -> 20123648 00010000:00010000:0.0:1789760716.712813:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.712825:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.712835:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.712841:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760716.712846:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.712854:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.712873:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.712880:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d8ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.712987:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32754us (33042us total) trans 4294972229 rc 0/0 00000100:00100000:0.0:1789760716.712998:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4941 00000100:00100000:0.0:1789760716.713008:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695079440970 time 695145505754 state 0x2 ref 0 00000100:00100000:1.0:1789760716.718403:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.718495:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625472 00000100:00100000:0.0:1789760716.718505:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625472 00000100:00100000:0.0:1789760716.718556:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695079440970 time 695145505754 state 0x0 ref 1 00000100:00100000:0.0:1789760716.718571:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695145766170 time 695145505754 state 0x1 ref 1 00000100:00100000:0.0:1789760716.718647:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695145766170 time 695183935309 state 0x0 ref 1 00000100:00100000:0.0:1789760716.718660:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760716.718731:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20123648, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:0.0:1789760716.718741:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.718760:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.718765:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.718772:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.718788:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.725627:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.725673:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.725702:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.725743:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625600 00000100:00100000:2.0:1789760716.725754:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625600 00000100:00100000:1.0:1789760716.725802:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625728 00000100:00100000:1.0:1789760716.725809:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625728 00000100:00100000:1.0:1789760716.725863:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625856 00000100:00100000:1.0:1789760716.725868:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625856 00000100:00100000:2.0:1789760716.725872:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 382 00000100:00100000:2.0:1789760716.725885:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92025fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515625600:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760716.725898:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 431 00000100:00100000:1.0:1789760716.725908:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515625728:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760716.725954:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92025fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515625600:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 72us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760716.725956:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515625728:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 48us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760716.725967:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 382 00000100:00100000:1.0:1789760716.725971:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 431 00000100:00100000:1.0:1789760716.725992:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 432 00000100:00100000:1.0:1789760716.726000:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515625856:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760716.726026:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515625856:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 28us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760716.726031:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 432 00000100:00080000:0.0:1789760716.726821:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760716.747333:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972230 is committed 00002000:00010000:2.0:1789760716.747380:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20123648 -> 20127744 00010000:00010000:2.0:1789760716.747393:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.747406:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.747413:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.747417:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.747420:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.747426:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.747441:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.747446:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d8c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.747516:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28855us (29121us total) trans 4294972230 rc 0/0 00000100:00100000:2.0:1789760716.747525:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4942 00000100:00100000:2.0:1789760716.747532:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695145766170 time 695183935309 state 0x2 ref 0 00000100:00100000:0.0:1789760716.750959:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760716.751037:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515625984 00000100:00100000:3.0:1789760716.751046:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515625984 00000100:00100000:3.0:1789760716.751102:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695145766170 time 695183935309 state 0x0 ref 1 00000100:00100000:3.0:1789760716.751118:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695184195725 time 695183935309 state 0x1 ref 1 00000100:00100000:3.0:1789760716.751149:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695184195725 time 695216438463 state 0x0 ref 1 00000100:00100000:3.0:1789760716.751160:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515625984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760716.751214:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20127744, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:3.0:1789760716.751223:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.751238:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760716.751242:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.751249:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.751263:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760716.785114:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20127744 -> 20131840 00010000:00010000:1.0:1789760716.785133:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.785147:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.785155:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.785163:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760716.785177:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.785186:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.785204:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.785214:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1d8c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.785344:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515625984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34181us (34389us total) trans 4294972231 rc 0/0 00000100:00100000:1.0:1789760716.785354:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4943 00000100:00100000:1.0:1789760716.785364:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695184195725 time 695216438463 state 0x2 ref 0 00000001:00080000:1.0:1789760716.785401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972231 is committed 00000100:00100000:1.0:1789760716.790932:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760716.791215:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626112 00000100:00100000:3.0:1789760716.791225:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626112 00000100:00100000:2.0:1789760716.791852:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695184195725 time 695216438463 state 0x0 ref 1 00000100:00100000:2.0:1789760716.791867:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695216698879 time 695216438463 state 0x1 ref 1 00000100:00100000:2.0:1789760716.791934:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695216698879 time 695257222740 state 0x0 ref 1 00000100:00100000:2.0:1789760716.791948:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.791998:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20131840, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:2.0:1789760716.792007:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.792023:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760716.792027:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.792033:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.792046:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760716.821220:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20131840 -> 20135936 00010000:00010000:3.0:1789760716.821240:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.821254:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.821260:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972232 is committed 00010000:00010000:3.0:1789760716.821262:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.821269:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760716.821273:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.821282:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.821301:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.821314:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1d8cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.821402:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29452us (30475us total) trans 4294972232 rc 0/0 00000100:00100000:3.0:1789760716.821411:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4944 00000100:00100000:3.0:1789760716.821420:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695216698879 time 695257222740 state 0x2 ref 0 00000100:00100000:2.0:1789760716.824096:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.824198:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626240 00000100:00100000:0.0:1789760716.824207:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626240 00000100:00100000:0.0:1789760716.824257:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695216698879 time 695257222740 state 0x0 ref 1 00000100:00100000:0.0:1789760716.824272:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695257483156 time 695257222740 state 0x1 ref 1 00000100:00100000:0.0:1789760716.824334:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695257483156 time 695289622401 state 0x0 ref 1 00000100:00100000:0.0:1789760716.824348:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760716.824392:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20135936, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa007 00010000:00010000:0.0:1789760716.824401:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.824416:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.824421:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.824428:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.824441:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.851245:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972233 is committed 00002000:00010000:1.0:1789760716.851437:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20135936 -> 20140032 00002000:00010000:1.0:1789760716.851441:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760716 -> 1789760717 00002000:00010000:1.0:1789760716.851444:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760716 -> 1789760717 00010000:00010000:1.0:1789760716.851456:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.851469:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.851477:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.851484:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760716.851488:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.851503:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.851521:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.851528:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d8d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.851624:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27274us (27532us total) trans 4294972233 rc 0/0 00000100:00100000:1.0:1789760716.851633:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4945 00000100:00100000:1.0:1789760716.851642:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695257483156 time 695289622401 state 0x2 ref 0 00000100:00100000:0.0:1789760716.858371:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.858462:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626368 00000100:00100000:2.0:1789760716.858472:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626368 00000100:00100000:2.0:1789760716.858525:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695257483156 time 695289622401 state 0x0 ref 1 00000100:00100000:2.0:1789760716.858541:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695289882817 time 695289622401 state 0x1 ref 1 00000100:00100000:2.0:1789760716.858572:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695289882817 time 695323861760 state 0x0 ref 1 00000100:00100000:2.0:1789760716.858586:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515626368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.858632:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20140032, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:2.0:1789760716.858641:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.858656:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760716.858660:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.858666:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.858680:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.885474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972234 is committed 00002000:00010000:2.0:1789760716.885556:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20140032 -> 20144128 00010000:00010000:2.0:1789760716.885571:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.885584:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.885599:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.885607:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.885611:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.885619:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.885638:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.885646:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.885777:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515626368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27182us (27404us total) trans 4294972234 rc 0/0 00000100:00100000:2.0:1789760716.885788:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4946 00000100:00100000:2.0:1789760716.885798:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695289882817 time 695323861760 state 0x2 ref 0 00000100:00100000:3.0:1789760716.890287:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760716.890373:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626496 00000100:00100000:0.0:1789760716.890382:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626496 00000100:00100000:0.0:1789760716.890433:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695289882817 time 695323861760 state 0x0 ref 1 00000100:00100000:0.0:1789760716.890447:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695324122176 time 695323861760 state 0x1 ref 1 00000100:00100000:0.0:1789760716.890516:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695324122176 time 695355803964 state 0x0 ref 1 00000100:00100000:0.0:1789760716.890528:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760716.890571:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20144128, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760716.890579:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.890594:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.890599:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.890604:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.890616:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.939859:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972235 is committed 00002000:00010000:2.0:1789760716.939923:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20144128 -> 20148224 00010000:00010000:2.0:1789760716.939939:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.939959:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.939968:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.939976:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760716.939981:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.939990:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.940009:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.940017:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d8e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.940115:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49583us (49832us total) trans 4294972235 rc 0/0 00000100:00100000:2.0:1789760716.940126:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4947 00000100:00100000:2.0:1789760716.940135:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695324122176 time 695355803964 state 0x2 ref 0 00000100:00100000:3.0:1789760716.943610:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760716.943701:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626624 00000100:00100000:2.0:1789760716.943739:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626624 00000100:00100000:2.0:1789760716.943793:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695324122176 time 695355803964 state 0x0 ref 1 00000100:00100000:2.0:1789760716.943816:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695356064380 time 695355803964 state 0x1 ref 1 00000100:00100000:2.0:1789760716.943846:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695356064380 time 695409135210 state 0x0 ref 1 00000100:00100000:2.0:1789760716.943858:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515626624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760716.943905:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20148224, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:2.0:1789760716.943914:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.943930:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760716.943934:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.943940:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.943953:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.991434:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20148224 -> 20152320 00010000:00010000:0.0:1789760716.991454:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.991467:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.991477:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.991495:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760716.991500:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.991509:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.991527:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.991546:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1d8eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.991633:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515626624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47771us (48026us total) trans 4294972236 rc 0/0 00000100:00100000:0.0:1789760716.991642:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4948 00000100:00100000:0.0:1789760716.991652:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695356064380 time 695409135210 state 0x2 ref 0 00000001:00080000:0.0:1789760716.991696:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972236 is committed 00000100:00100000:0.0:1789760716.995216:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760716.995299:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626752 00000100:00100000:1.0:1789760716.995310:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626752 00000100:00100000:1.0:1789760716.995363:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695356064380 time 695409135210 state 0x0 ref 1 00000100:00100000:1.0:1789760716.995378:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695409395626 time 695409135210 state 0x1 ref 1 00000100:00100000:1.0:1789760716.995410:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695409395626 time 695460699208 state 0x0 ref 1 00000100:00100000:1.0:1789760716.995422:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760716.995468:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20152320, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:1.0:1789760716.995477:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.995493:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.995497:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.995504:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.995517:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.033523:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972237 is committed 00002000:00010000:3.0:1789760717.033523:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20152320 -> 20156416 00010000:00010000:3.0:1789760717.033542:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.033554:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.033572:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.033580:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760717.033585:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.033594:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.033629:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.033638:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1d8f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.033884:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38458us (38671us total) trans 4294972237 rc 0/0 00000100:00100000:3.0:1789760717.033894:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4949 00000100:00100000:3.0:1789760717.033903:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695409395626 time 695460699208 state 0x2 ref 0 00000100:00100000:3.0:1789760717.038779:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760717.038838:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515626880 00000100:00100000:2.0:1789760717.038849:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515626880 00000100:00100000:2.0:1789760717.038899:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695409395626 time 695460699208 state 0x0 ref 1 00000100:00100000:2.0:1789760717.038916:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695460959624 time 695460699208 state 0x1 ref 1 00000100:00100000:2.0:1789760717.038938:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695460959624 time 695504226906 state 0x0 ref 1 00000100:00100000:2.0:1789760717.038952:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515626880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760717.038999:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20156416, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:2.0:1789760717.039008:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.039024:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760717.039029:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.039035:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.039049:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760717.104140:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20156416 -> 20160512 00010000:00010000:3.0:1789760717.104159:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.104183:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.104192:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.104201:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760717.104206:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.104219:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.104238:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.104246:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1d8f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.104336:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515626880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65383us (65562us total) trans 4294972238 rc 0/0 00000100:00100000:3.0:1789760717.104345:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4950 00000100:00100000:3.0:1789760717.104355:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695460959624 time 695504226906 state 0x2 ref 0 00000001:00080000:0.0:1789760717.105270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972238 is committed 00000100:00100000:2.0:1789760717.107880:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760717.108029:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627008 00000100:00100000:3.0:1789760717.108038:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627008 00000100:00100000:3.0:1789760717.108101:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695460959624 time 695504226906 state 0x0 ref 1 00000100:00100000:3.0:1789760717.108117:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695504487322 time 695504226906 state 0x1 ref 1 00000100:00100000:3.0:1789760717.108140:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695504487322 time 695573429305 state 0x0 ref 1 00000100:00100000:3.0:1789760717.108153:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515627008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760717.108215:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20160512, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:3.0:1789760717.108224:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.108242:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760717.108246:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.108253:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.108267:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.120935:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760717.173518:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972239 is committed 00002000:00010000:3.0:1789760717.173607:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20160512 -> 20164608 00010000:00010000:3.0:1789760717.173622:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173635:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173644:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173652:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760717.173657:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173667:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173686:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.173695:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d900 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.173846:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515627008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65690us (65970us total) trans 4294972239 rc 0/0 00000100:00100000:3.0:1789760717.173856:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4951 00000100:00100000:3.0:1789760717.173867:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695504487322 time 695573429305 state 0x2 ref 0 00000100:00100000:2.0:1789760717.177425:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.177579:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627136 00000100:00100000:0.0:1789760717.177589:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627136 00000100:00100000:0.0:1789760717.177640:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695504487322 time 695573429305 state 0x0 ref 1 00000100:00100000:0.0:1789760717.177656:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695573689721 time 695573429305 state 0x1 ref 1 00000100:00100000:0.0:1789760717.177749:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695573689721 time 695643036624 state 0x0 ref 1 00000100:00100000:0.0:1789760717.177763:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515627136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.177907:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20164608, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.177918:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.177935:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760717.177940:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.177946:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.177960:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.233481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972240 is committed 00002000:00010000:0.0:1789760717.233652:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20164608 -> 20168704 00010000:00010000:0.0:1789760717.233669:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233682:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233692:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233700:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.233734:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233744:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233764:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233772:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1d907 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.233876:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515627136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56111us (56455us total) trans 4294972240 rc 0/0 00000100:00100000:0.0:1789760717.233886:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4952 00000100:00100000:0.0:1789760717.233896:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695573689721 time 695643036624 state 0x2 ref 0 00000100:00100000:1.0:1789760717.237414:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.237549:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627264 00000100:00100000:0.0:1789760717.237559:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627264 00000100:00100000:0.0:1789760717.237611:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695573689721 time 695643036624 state 0x0 ref 1 00000100:00100000:0.0:1789760717.237627:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695643297040 time 695643036624 state 0x1 ref 1 00000100:00100000:0.0:1789760717.237659:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695643297040 time 695702948326 state 0x0 ref 1 00000100:00100000:0.0:1789760717.237672:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515627264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.237746:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20168704, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.237754:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.237766:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760717.237769:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.237774:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.237784:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.279047:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972241 is committed 00002000:00010000:2.0:1789760717.279499:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20168704 -> 20172800 00010000:00010000:2.0:1789760717.279514:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.279527:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.279535:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.279548:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760717.279553:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.279562:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.279580:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.279587:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d90e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.279677:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515627264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42002us (42269us total) trans 4294972241 rc 0/0 00000100:00100000:2.0:1789760717.279687:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4953 00000100:00100000:2.0:1789760717.279697:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695643297040 time 695702948326 state 0x2 ref 0 00000100:00100000:2.0:1789760717.284418:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.284498:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627392 00000100:00100000:0.0:1789760717.284508:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627392 00000100:00100000:0.0:1789760717.284562:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695643297040 time 695702948326 state 0x0 ref 1 00000100:00100000:0.0:1789760717.284578:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695703208742 time 695702948326 state 0x1 ref 1 00000100:00100000:0.0:1789760717.284610:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695703208742 time 695749899120 state 0x0 ref 1 00000100:00100000:0.0:1789760717.284624:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515627392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.284678:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20172800, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.284688:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.284737:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760717.284743:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.284750:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.284764:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760717.315928:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20172800 -> 20176896 00010000:00010000:1.0:1789760717.315948:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.315962:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.315972:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.315981:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760717.315985:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.315995:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.316020:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.316027:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1d915 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.316307:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515627392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31680us (32105us total) trans 4294972242 rc 0/0 00000100:00100000:1.0:1789760717.316319:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4954 00000100:00100000:1.0:1789760717.316328:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695703208742 time 695749899120 state 0x2 ref 0 00000001:00080000:3.0:1789760717.316737:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972242 is committed 00000100:00100000:0.0:1789760717.320452:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760717.320975:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627520 00000100:00100000:2.0:1789760717.320985:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627520 00000100:00100000:2.0:1789760717.321035:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695703208742 time 695749899120 state 0x0 ref 1 00000100:00100000:2.0:1789760717.321049:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695750159536 time 695749899120 state 0x1 ref 1 00000100:00100000:2.0:1789760717.321079:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695750159536 time 695786368688 state 0x0 ref 1 00000100:00100000:2.0:1789760717.321104:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515627520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760717.321150:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20176896, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:2.0:1789760717.321160:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.321186:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760717.321191:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.321196:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.321364:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.350041:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972243 is committed 00002000:00010000:2.0:1789760717.350593:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20176896 -> 20180992 00010000:00010000:2.0:1789760717.350609:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.350623:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.350631:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.350640:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760717.350644:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.350653:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.350671:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.350678:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1d91c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.351270:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515627520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29981us (30639us total) trans 4294972243 rc 0/0 00000100:00100000:3.0:1789760717.351282:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4955 00000100:00100000:3.0:1789760717.351292:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695750159536 time 695786368688 state 0x2 ref 0 00000100:00100000:2.0:1789760717.358040:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760717.358181:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627648 00000100:00100000:3.0:1789760717.358192:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627648 00000100:00100000:3.0:1789760717.358251:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695750159536 time 695786368688 state 0x0 ref 1 00000100:00100000:3.0:1789760717.358266:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695786629104 time 695786368688 state 0x1 ref 1 00000100:00100000:3.0:1789760717.358299:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695786629104 time 695823588353 state 0x0 ref 1 00000100:00100000:3.0:1789760717.358311:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515627648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760717.358368:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20180992, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:3.0:1789760717.358378:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.358395:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760717.358399:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.358405:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.358419:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.389366:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972244 is committed 00002000:00010000:0.0:1789760717.389476:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20180992 -> 20185088 00010000:00010000:0.0:1789760717.389491:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.389504:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.389513:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.389521:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.389526:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.389535:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.389554:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.389563:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d923 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.389664:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515627648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31351us (31629us total) trans 4294972244 rc 0/0 00000100:00100000:0.0:1789760717.389674:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4956 00000100:00100000:0.0:1789760717.389682:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695786629104 time 695823588353 state 0x2 ref 0 00000100:00100000:2.0:1789760717.394058:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760717.394268:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627776 00000100:00100000:3.0:1789760717.394278:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627776 00000100:00100000:3.0:1789760717.394330:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695786629104 time 695823588353 state 0x0 ref 1 00000100:00100000:3.0:1789760717.394344:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695823848769 time 695823588353 state 0x1 ref 1 00000100:00100000:3.0:1789760717.394416:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695823848769 time 695859704356 state 0x0 ref 1 00000100:00100000:3.0:1789760717.394429:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515627776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760717.394471:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20185088, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:3.0:1789760717.394479:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.394494:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760717.394498:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.394504:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.394522:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760717.417284:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20185088 -> 20189184 00010000:00010000:0.0:1789760717.417304:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.417318:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.417329:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.417337:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.417342:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.417356:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.417376:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.417383:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d92a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.417527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972245 is committed 00000100:00100000:0.0:1789760717.417592:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515627776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23160us (23539us total) trans 4294972245 rc 0/0 00000100:00100000:0.0:1789760717.417603:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4957 00000100:00100000:0.0:1789760717.417613:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695823848769 time 695859704356 state 0x2 ref 0 00000100:00100000:1.0:1789760717.421043:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.421128:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515627904 00000100:00100000:0.0:1789760717.421138:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515627904 00000100:00100000:0.0:1789760717.421204:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695823848769 time 695859704356 state 0x0 ref 1 00000100:00100000:0.0:1789760717.421220:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695859964772 time 695859704356 state 0x1 ref 1 00000100:00100000:0.0:1789760717.421246:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695859964772 time 695886535272 state 0x0 ref 1 00000100:00100000:0.0:1789760717.421259:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515627904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.421302:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20189184, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.421310:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.421325:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760717.421329:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.421335:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.421348:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.452562:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972246 is committed 00002000:00010000:0.0:1789760717.452859:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20189184 -> 20193280 00010000:00010000:0.0:1789760717.452875:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.452888:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.452897:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.452904:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.452908:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.452918:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.452937:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.452945:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d931 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.453028:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515627904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31767us (31990us total) trans 4294972246 rc 0/0 00000100:00100000:0.0:1789760717.453039:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4958 00000100:00100000:0.0:1789760717.453049:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695859964772 time 695886535272 state 0x2 ref 0 00000100:00100000:2.0:1789760717.461301:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760717.461462:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628032 00000100:00100000:3.0:1789760717.461473:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628032 00000100:00100000:3.0:1789760717.461539:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695859964772 time 695886535272 state 0x0 ref 1 00000100:00100000:3.0:1789760717.461556:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695886795688 time 695886535272 state 0x1 ref 1 00000100:00100000:3.0:1789760717.461587:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695886795688 time 695926876006 state 0x0 ref 1 00000100:00100000:3.0:1789760717.461608:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760717.461658:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20193280, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:3.0:1789760717.461667:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.461682:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760717.461687:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.461693:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.461745:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.485150:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972247 is committed 00002000:00010000:0.0:1789760717.485247:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20193280 -> 20197376 00010000:00010000:0.0:1789760717.485263:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.485277:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.485286:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.485294:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.485298:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.485308:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.485334:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.485342:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d938 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.485634:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23826us (24144us total) trans 4294972247 rc 0/0 00000100:00100000:0.0:1789760717.485644:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4959 00000100:00100000:0.0:1789760717.485654:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695886795688 time 695926876006 state 0x2 ref 0 00000100:00100000:3.0:1789760717.488624:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760717.488685:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628160 00000100:00100000:1.0:1789760717.488695:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628160 00000100:00100000:1.0:1789760717.488774:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695886795688 time 695926876006 state 0x0 ref 1 00000100:00100000:1.0:1789760717.488787:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695927136422 time 695926876006 state 0x1 ref 1 00000100:00100000:1.0:1789760717.488899:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695927136422 time 695954187415 state 0x0 ref 1 00000100:00100000:1.0:1789760717.488913:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515628160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760717.488957:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20197376, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:1.0:1789760717.488966:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.488982:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760717.488988:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.488994:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.489006:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.513832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972248 is committed 00002000:00010000:0.0:1789760717.514029:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20197376 -> 20201472 00010000:00010000:0.0:1789760717.514043:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.514056:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.514065:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.514073:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.514078:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.514224:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.514244:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.514252:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1d93f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.514385:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515628160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25462us (25754us total) trans 4294972248 rc 0/0 00000100:00100000:0.0:1789760717.514397:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4960 00000100:00100000:0.0:1789760717.514406:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695927136422 time 695954187415 state 0x2 ref 0 00000100:00100000:1.0:1789760717.518820:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760717.519112:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628288 00000100:00100000:3.0:1789760717.519122:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628288 00000100:00100000:3.0:1789760717.519183:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695927136422 time 695954187415 state 0x0 ref 1 00000100:00100000:3.0:1789760717.519198:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695954447831 time 695954187415 state 0x1 ref 1 00000100:00100000:3.0:1789760717.519229:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695954447831 time 695984518176 state 0x0 ref 1 00000100:00100000:3.0:1789760717.519241:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515628288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760717.519287:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20201472, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:3.0:1789760717.519296:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.519312:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760717.519316:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.519324:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.519338:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760717.563390:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20201472 -> 20205568 00010000:00010000:0.0:1789760717.563408:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.563421:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.563429:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.563435:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.563441:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.563451:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.563483:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.563490:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d946 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.563578:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515628288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44335us (44770us total) trans 4294972249 rc 0/0 00000100:00100000:0.0:1789760717.563589:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4961 00000100:00100000:0.0:1789760717.563599:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695954447831 time 695984518176 state 0x2 ref 0 00000001:00080000:1.0:1789760717.563671:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972249 is committed 00000100:00100000:2.0:1789760717.568051:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.568139:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628416 00000100:00100000:0.0:1789760717.568148:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628416 00000100:00100000:0.0:1789760717.568214:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695954447831 time 695984518176 state 0x0 ref 1 00000100:00100000:0.0:1789760717.568229:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695984778592 time 695984518176 state 0x1 ref 1 00000100:00100000:0.0:1789760717.568261:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695984778592 time 696033550104 state 0x0 ref 1 00000100:00100000:0.0:1789760717.568274:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515628416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.568323:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20205568, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.568331:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.568348:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.568352:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.568359:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.568372:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.597623:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972250 is committed 00002000:00010000:3.0:1789760717.597797:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20205568 -> 20209664 00010000:00010000:3.0:1789760717.597898:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.597917:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.597926:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.597932:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760717.597937:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.597946:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.597962:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.597969:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1d94d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.598079:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515628416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29803us (30034us total) trans 4294972250 rc 0/0 00000100:00100000:3.0:1789760717.598094:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4962 00000100:00100000:3.0:1789760717.598104:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695984778592 time 696033550104 state 0x2 ref 0 00000100:00100000:3.0:1789760717.605579:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.605740:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628544 00000100:00100000:0.0:1789760717.605751:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628544 00000100:00100000:0.0:1789760717.605815:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 695984778592 time 696033550104 state 0x0 ref 1 00000100:00100000:0.0:1789760717.605831:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696033810520 time 696033550104 state 0x1 ref 1 00000100:00100000:0.0:1789760717.605868:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696033810520 time 696071156379 state 0x0 ref 1 00000100:00100000:0.0:1789760717.605881:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.605927:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20209664, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.605936:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.605952:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.605957:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.605963:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.605977:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.676651:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972251 is committed 00002000:00010000:0.0:1789760717.676772:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20209664 -> 20213760 00010000:00010000:0.0:1789760717.676788:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.676800:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.676822:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.676830:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.676836:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.676845:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.676867:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.676875:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1d954 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.676973:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71090us (71399us total) trans 4294972251 rc 0/0 00000100:00100000:0.0:1789760717.676983:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4963 00000100:00100000:0.0:1789760717.676993:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696033810520 time 696071156379 state 0x2 ref 0 00000100:00100000:1.0:1789760717.688805:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.688902:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628672 00000100:00100000:0.0:1789760717.688912:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628672 00000100:00100000:0.0:1789760717.688966:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696033810520 time 696071156379 state 0x0 ref 1 00000100:00100000:0.0:1789760717.688982:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696071416795 time 696071156379 state 0x1 ref 1 00000100:00100000:0.0:1789760717.689016:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696071416795 time 696154305492 state 0x0 ref 1 00000100:00100000:0.0:1789760717.689029:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515628672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.689079:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20213760, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.689088:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.689104:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.689109:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.689114:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.689129:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.756313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972252 is committed 00002000:00010000:0.0:1789760717.756461:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20213760 -> 20217856 00010000:00010000:0.0:1789760717.756481:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.756496:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.756505:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.756513:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760717.756519:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.756530:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.756554:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.756561:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d95b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.756694:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515628672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67649us (67881us total) trans 4294972252 rc 0/0 00000100:00100000:0.0:1789760717.756762:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4964 00000100:00100000:0.0:1789760717.756774:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696071416795 time 696154305492 state 0x2 ref 0 00000100:00100000:3.0:1789760717.762783:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760717.763000:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628800 00000100:00100000:0.0:1789760717.763012:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628800 00000100:00100000:0.0:1789760717.763092:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696071416795 time 696154305492 state 0x0 ref 1 00000100:00100000:0.0:1789760717.763112:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696154565908 time 696154305492 state 0x1 ref 1 00000100:00100000:0.0:1789760717.763144:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696154565908 time 696228433237 state 0x0 ref 1 00000100:00100000:0.0:1789760717.763167:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760717.763232:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20217856, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:0.0:1789760717.763242:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.763260:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.763265:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.763273:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.763290:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.862630:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972253 is committed 00002000:00010000:3.0:1789760717.862916:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20217856 -> 20221952 00010000:00010000:3.0:1789760717.862938:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.862954:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.862965:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.862974:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760717.862980:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.862991:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.863016:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.863024:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1d962 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.863153:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 99989us (100376us total) trans 4294972253 rc 0/0 00000100:00100000:3.0:1789760717.863178:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4965 00000100:00100000:3.0:1789760717.863191:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696154565908 time 696228433237 state 0x2 ref 0 00000100:00100000:3.0:1789760717.870325:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760717.870537:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515628928 00000100:00100000:1.0:1789760717.870551:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515628928 00000100:00100000:1.0:1789760717.870627:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696154565908 time 696228433237 state 0x0 ref 1 00000100:00100000:1.0:1789760717.870647:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696228693653 time 696228433237 state 0x1 ref 1 00000100:00100000:1.0:1789760717.870684:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696228693653 time 696335973191 state 0x0 ref 1 00000100:00100000:1.0:1789760717.870699:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760717.870811:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20221952, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xa007 00010000:00010000:1.0:1789760717.870822:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.870842:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760717.870847:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.870855:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.870872:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.954488:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972254 is committed 00002000:00010000:1.0:1789760717.954596:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20221952 -> 20226048 00002000:00010000:1.0:1789760717.954601:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760717 -> 1789760718 00002000:00010000:1.0:1789760717.954604:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760717 -> 1789760718 00010000:00010000:1.0:1789760717.954690:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.954779:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.954790:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.954798:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760717.954804:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.954814:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.954839:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.954849:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d969 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.955023:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515628928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 84319us (84707us total) trans 4294972254 rc 0/0 00000100:00100000:1.0:1789760717.955035:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4966 00000100:00100000:1.0:1789760717.955048:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696228693653 time 696335973191 state 0x2 ref 0 00000100:00100000:0.0:1789760717.960900:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760717.961069:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629056 00000100:00100000:3.0:1789760717.961146:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629056 00000100:00100000:3.0:1789760717.961290:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696228693653 time 696335973191 state 0x0 ref 1 00000100:00100000:3.0:1789760717.961310:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696336233607 time 696335973191 state 0x1 ref 1 00000100:00100000:3.0:1789760717.961438:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696336233607 time 696426725654 state 0x0 ref 1 00000100:00100000:3.0:1789760717.961453:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760717.961518:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20226048, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:3.0:1789760717.961530:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.961548:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760717.961553:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.961560:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.961575:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.998936:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972255 is committed 00002000:00010000:3.0:1789760717.999190:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20226048 -> 20230144 00010000:00010000:3.0:1789760717.999214:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.999229:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.999238:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.999246:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760717.999252:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.999263:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.999288:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.999297:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d970 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.999523:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38065us (38629us total) trans 4294972255 rc 0/0 00000100:00100000:3.0:1789760717.999537:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4967 00000100:00100000:3.0:1789760717.999549:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696336233607 time 696426725654 state 0x2 ref 0 00000100:00100000:1.0:1789760718.003467:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760718.006040:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629184 00000100:00100000:3.0:1789760718.006064:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629184 00000100:00100000:2.0:1789760718.006389:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696336233607 time 696426725654 state 0x0 ref 1 00000100:00100000:2.0:1789760718.006410:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696426986070 time 696426725654 state 0x1 ref 1 00000100:00100000:2.0:1789760718.006474:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696426986070 time 696471762089 state 0x0 ref 1 00000100:00100000:2.0:1789760718.006492:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515629184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760718.006560:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20230144, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:2.0:1789760718.006573:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.006594:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760718.006599:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.006608:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.006628:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760718.010757:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760718.047443:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20230144 -> 20234240 00010000:00010000:3.0:1789760718.047471:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.047488:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.047498:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.047506:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760718.047511:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.047521:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.047546:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.047554:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1d977 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.047680:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515629184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41184us (44219us total) trans 4294972256 rc 0/0 00000100:00100000:3.0:1789760718.047696:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4968 00000100:00100000:3.0:1789760718.047749:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696426986070 time 696471762089 state 0x2 ref 0 00000001:00080000:1.0:1789760718.050256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972256 is committed 00000100:00100000:1.0:1789760718.050925:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.051038:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629312 00000100:00100000:0.0:1789760718.051053:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629312 00000100:00100000:0.0:1789760718.051198:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696426986070 time 696471762089 state 0x0 ref 1 00000100:00100000:0.0:1789760718.051217:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696472022505 time 696471762089 state 0x1 ref 1 00000100:00100000:0.0:1789760718.051283:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696472022505 time 696516571047 state 0x0 ref 1 00000100:00100000:0.0:1789760718.051297:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.051363:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20234240, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.051377:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.051397:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760718.051402:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.051410:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.051428:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.083864:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972257 is committed 00002000:00010000:0.0:1789760718.083984:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20234240 -> 20238336 00010000:00010000:0.0:1789760718.084003:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.084018:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.084029:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.084038:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.084043:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.084053:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.084138:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.084149:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1d97e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.084315:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33012us (33391us total) trans 4294972257 rc 0/0 00000100:00100000:0.0:1789760718.084327:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4969 00000100:00100000:0.0:1789760718.084340:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696472022505 time 696516571047 state 0x2 ref 0 00000100:00100000:0.0:1789760718.087857:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760718.087957:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629440 00000100:00100000:1.0:1789760718.087970:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629440 00000100:00100000:1.0:1789760718.088043:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696472022505 time 696516571047 state 0x0 ref 1 00000100:00100000:1.0:1789760718.088063:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696516831463 time 696516571047 state 0x1 ref 1 00000100:00100000:1.0:1789760718.088193:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696516831463 time 696553480180 state 0x0 ref 1 00000100:00100000:1.0:1789760718.088208:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760718.088271:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20238336, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:1.0:1789760718.088281:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.088305:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760718.088310:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.088317:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.088332:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.118540:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972258 is committed 00002000:00010000:3.0:1789760718.118625:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20238336 -> 20242432 00010000:00010000:3.0:1789760718.118647:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.118661:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.118670:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.118678:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760718.118684:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.118694:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.118743:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.118754:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1d985 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.118949:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30736us (31232us total) trans 4294972258 rc 0/0 00000100:00100000:3.0:1789760718.118963:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4970 00000100:00100000:3.0:1789760718.118976:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696516831463 time 696553480180 state 0x2 ref 0 00000100:00100000:1.0:1789760718.121902:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760718.121995:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629568 00000100:00100000:3.0:1789760718.122009:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629568 00000100:00100000:3.0:1789760718.122076:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696516831463 time 696553480180 state 0x0 ref 1 00000100:00100000:3.0:1789760718.122171:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696553740596 time 696553480180 state 0x1 ref 1 00000100:00100000:3.0:1789760718.122245:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696553740596 time 696587532815 state 0x0 ref 1 00000100:00100000:3.0:1789760718.122258:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760718.122321:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20242432, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:3.0:1789760718.122332:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.122351:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760718.122356:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.122363:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.122380:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.142969:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760718.152037:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972259 is committed 00002000:00010000:0.0:1789760718.152226:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20242432 -> 20246528 00010000:00010000:0.0:1789760718.152246:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.152268:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.152280:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.152288:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.152294:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.152305:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.152330:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.152339:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1d98c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.152452:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30189us (30557us total) trans 4294972259 rc 0/0 00000100:00100000:0.0:1789760718.152464:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4971 00000100:00100000:0.0:1789760718.152477:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696553740596 time 696587532815 state 0x2 ref 0 00000100:00100000:1.0:1789760718.155539:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.155632:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629696 00000100:00100000:0.0:1789760718.155645:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629696 00000100:00100000:0.0:1789760718.155749:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696553740596 time 696587532815 state 0x0 ref 1 00000100:00100000:0.0:1789760718.155770:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696587793231 time 696587532815 state 0x1 ref 1 00000100:00100000:0.0:1789760718.155799:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696587793231 time 696621088220 state 0x0 ref 1 00000100:00100000:0.0:1789760718.155820:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515629696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.155885:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20246528, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.155896:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.155917:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760718.155922:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.155930:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.155947:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.216112:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972260 is committed 00002000:00010000:2.0:1789760718.216480:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20246528 -> 20250624 00010000:00010000:2.0:1789760718.216505:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216520:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216531:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216538:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760718.216543:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216553:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216583:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.216590:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d993 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.217631:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515629696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61801us (62093us total) trans 4294972260 rc 0/0 00000100:00100000:2.0:1789760718.217647:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4972 00000100:00100000:2.0:1789760718.217660:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696587793231 time 696621088220 state 0x2 ref 0 00000100:00100000:0.0:1789760718.221773:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760718.229779:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629824 00000100:00100000:3.0:1789760718.229794:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629824 00000100:00100000:3.0:1789760718.229858:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696587793231 time 696621088220 state 0x0 ref 1 00000100:00100000:3.0:1789760718.229874:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696621348636 time 696621088220 state 0x1 ref 1 00000100:00100000:3.0:1789760718.229908:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696621348636 time 696695197426 state 0x0 ref 1 00000100:00100000:3.0:1789760718.229923:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760718.229972:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20250624, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:3.0:1789760718.229981:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.229996:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760718.230001:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.230007:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.230022:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.263393:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972261 is committed 00002000:00010000:0.0:1789760718.263489:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20250624 -> 20254720 00010000:00010000:0.0:1789760718.263565:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.263578:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.263591:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.263599:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.263604:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.263614:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.263632:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.263640:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1d99a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.263800:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515629824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33876us (42130us total) trans 4294972261 rc 0/0 00000100:00100000:0.0:1789760718.263810:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4973 00000100:00100000:0.0:1789760718.263820:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696621348636 time 696695197426 state 0x2 ref 0 00000100:00100000:3.0:1789760718.277622:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760718.281181:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515629952 00000100:00100000:2.0:1789760718.281196:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515629952 00000100:00100000:3.0:1789760718.285268:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696621348636 time 696695197426 state 0x0 ref 1 00000100:00100000:3.0:1789760718.285290:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696695457842 time 696695197426 state 0x1 ref 1 00000100:00100000:3.0:1789760718.285374:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696695457842 time 696750662769 state 0x0 ref 1 00000100:00100000:3.0:1789760718.285389:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515629952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760718.285440:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20254720, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:3.0:1789760718.285449:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.285465:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760718.285470:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.285476:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.285491:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.334151:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae802f81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544407552:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760718.334313:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.334434:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407552 00000100:00100000:3.0:1789760718.334443:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407552 00000100:00100000:3.0:1789760718.334504:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1789760718.334515:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544407552:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760718.334549:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760718.334594:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544407552:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.334601:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00080000:2.0:1789760718.335002:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760718.335040:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae802f81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544407552:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760718.336627:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80445c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544407680:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760718.336681:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760718.337177:0:11703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407680 00000100:00100000:3.0:1789760718.337254:0:11703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407680 00000100:00100000:3.0:1789760718.337315:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789760718.337324:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544407680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760718.337342:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3837788160 avail 3823108096 00000100:00080000:0.0:1789760718.337405:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760718.343995:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80445c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544407680:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760718.343996:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544407680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 6653us (7295us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760718.344013:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00002000:00010000:0.0:1789760718.346138:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20254720 -> 20258816 00010000:00010000:0.0:1789760718.346169:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.346190:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.346199:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.346205:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.346210:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.346218:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.346237:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.346246:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1d9a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.346335:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515629952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60945us (68718us total) trans 4294972262 rc 0/0 00000100:00100000:0.0:1789760718.346347:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4974 00000100:00100000:0.0:1789760718.346355:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696695457842 time 696750662769 state 0x2 ref 0 00000001:00080000:3.0:1789760718.346629:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972262 is committed 00000100:00100000:1.0:1789760718.361178:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760718.364798:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630080 00000100:00100000:3.0:1789760718.364812:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630080 00000100:00100000:3.0:1789760718.364874:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696695457842 time 696750662769 state 0x0 ref 1 00000100:00100000:3.0:1789760718.364890:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696750923185 time 696750662769 state 0x1 ref 1 00000100:00100000:3.0:1789760718.364924:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696750923185 time 696830213582 state 0x0 ref 1 00000100:00100000:3.0:1789760718.364937:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760718.364984:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20258816, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:3.0:1789760718.364992:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.365007:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760718.365011:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.365018:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.365037:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.440401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972263 is committed 00002000:00010000:0.0:1789760718.440747:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20258816 -> 20262912 00010000:00010000:0.0:1789760718.440764:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.440777:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.440787:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.440795:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.440800:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.440808:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.440827:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.440834:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1d9a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.440927:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75987us (79754us total) trans 4294972263 rc 0/0 00000100:00100000:0.0:1789760718.440936:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4975 00000100:00100000:0.0:1789760718.440946:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696750923185 time 696830213582 state 0x2 ref 0 00000100:00100000:2.0:1789760718.445194:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760718.446368:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630208 00000100:00100000:1.0:1789760718.446379:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630208 00000100:00100000:1.0:1789760718.446558:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696750923185 time 696830213582 state 0x0 ref 1 00000100:00100000:1.0:1789760718.446576:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696830473998 time 696830213582 state 0x1 ref 1 00000100:00100000:1.0:1789760718.446607:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696830473998 time 696911895539 state 0x0 ref 1 00000100:00100000:1.0:1789760718.446621:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515630208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760718.446669:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20262912, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:1.0:1789760718.446679:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.446693:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760718.446697:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.446704:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.446751:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.475276:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20262912 -> 20267008 00010000:00010000:3.0:1789760718.475295:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.475309:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.475318:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.475326:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760718.475487:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.475500:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.475521:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.475529:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1d9af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.475620:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515630208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28997us (30429us total) trans 4294972264 rc 0/0 00000100:00100000:3.0:1789760718.475631:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4976 00000100:00100000:3.0:1789760718.475641:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696830473998 time 696911895539 state 0x2 ref 0 00000001:00080000:0.0:1789760718.475698:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972264 is committed 00000100:00100000:3.0:1789760718.480163:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.480260:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630336 00000100:00100000:0.0:1789760718.480269:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630336 00000100:00100000:0.0:1789760718.480337:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696830473998 time 696911895539 state 0x0 ref 1 00000100:00100000:0.0:1789760718.480358:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696912155955 time 696911895539 state 0x1 ref 1 00000100:00100000:0.0:1789760718.480387:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696912155955 time 696945676501 state 0x0 ref 1 00000100:00100000:0.0:1789760718.480403:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.480458:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20267008, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.480467:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.480481:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760718.480485:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.480491:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.480505:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.504189:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972265 is committed 00002000:00010000:0.0:1789760718.504266:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20267008 -> 20271104 00010000:00010000:0.0:1789760718.504281:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504300:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504308:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504313:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.504316:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504322:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504337:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504351:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1d9b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.504859:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24028us (24299us total) trans 4294972265 rc 0/0 00000100:00100000:0.0:1789760718.504871:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4977 00000100:00100000:0.0:1789760718.504880:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696912155955 time 696945676501 state 0x2 ref 0 00000100:00100000:1.0:1789760718.508266:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.508346:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630464 00000100:00100000:0.0:1789760718.508357:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630464 00000100:00100000:0.0:1789760718.508409:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696912155955 time 696945676501 state 0x0 ref 1 00000100:00100000:0.0:1789760718.508424:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696945936917 time 696945676501 state 0x1 ref 1 00000100:00100000:0.0:1789760718.508450:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696945936917 time 696973739653 state 0x0 ref 1 00000100:00100000:0.0:1789760718.508462:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515630464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.508508:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20271104, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.508518:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.508534:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760718.508539:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.508545:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.508558:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.539388:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972266 is committed 00002000:00010000:0.0:1789760718.539433:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20271104 -> 20275200 00010000:00010000:0.0:1789760718.539449:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.539461:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.539476:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.539484:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.539489:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.539498:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.539518:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.539523:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.539736:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515630464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31244us (31449us total) trans 4294972266 rc 0/0 00000100:00100000:0.0:1789760718.539746:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4978 00000100:00100000:0.0:1789760718.539755:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696945936917 time 696973739653 state 0x2 ref 0 00000100:00100000:3.0:1789760718.543011:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.544224:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630592 00000100:00100000:0.0:1789760718.544237:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630592 00000100:00100000:0.0:1789760718.544297:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696945936917 time 696973739653 state 0x0 ref 1 00000100:00100000:0.0:1789760718.544313:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696974000069 time 696973739653 state 0x1 ref 1 00000100:00100000:0.0:1789760718.544358:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696974000069 time 697009646328 state 0x0 ref 1 00000100:00100000:0.0:1789760718.544370:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.544425:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20275200, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.544634:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.544650:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.544654:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.544661:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.544675:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760718.564662:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20275200 -> 20279296 00000001:00080000:1.0:1789760718.564677:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972267 is committed 00010000:00010000:0.0:1789760718.564682:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.564694:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.564729:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.564738:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.564744:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.564754:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.564774:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.564781:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.564858:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 20486us (21854us total) trans 4294972267 rc 0/0 00000100:00100000:0.0:1789760718.564869:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4979 00000100:00100000:0.0:1789760718.564878:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696974000069 time 697009646328 state 0x2 ref 0 00000100:00100000:3.0:1789760718.574775:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.575272:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630720 00000100:00100000:0.0:1789760718.575282:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630720 00000100:00100000:0.0:1789760718.575340:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 696974000069 time 697009646328 state 0x0 ref 1 00000100:00100000:0.0:1789760718.575553:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697009906744 time 697009646328 state 0x1 ref 1 00000100:00100000:0.0:1789760718.575588:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697009906744 time 697040876903 state 0x0 ref 1 00000100:00100000:0.0:1789760718.575600:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515630720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.575646:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20279296, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.575654:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.575670:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.575674:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.575679:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.575693:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760718.611934:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20279296 -> 20283392 00010000:00010000:0.0:1789760718.611957:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.611970:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.611979:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.611986:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.611991:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.612001:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.612033:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.612042:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.612165:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515630720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36551us (37382us total) trans 4294972268 rc 0/0 00000100:00100000:0.0:1789760718.612177:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4980 00000100:00100000:0.0:1789760718.612188:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697009906744 time 697040876903 state 0x2 ref 0 00000001:00080000:1.0:1789760718.612273:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972268 is committed 00000100:00100000:1.0:1789760718.620174:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760718.620251:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630848 00000100:00100000:3.0:1789760718.620261:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630848 00000100:00100000:3.0:1789760718.620310:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697009906744 time 697040876903 state 0x0 ref 1 00000100:00100000:3.0:1789760718.620325:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697041137319 time 697040876903 state 0x1 ref 1 00000100:00100000:3.0:1789760718.620395:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697041137319 time 697085684492 state 0x0 ref 1 00000100:00100000:3.0:1789760718.620407:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760718.620457:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20283392, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:3.0:1789760718.620466:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.620481:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760718.620485:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.620491:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.620505:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.643289:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972269 is committed 00002000:00010000:0.0:1789760718.643589:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20283392 -> 20287488 00010000:00010000:0.0:1789760718.643605:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.643618:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.643627:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.643634:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.643639:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.643653:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.643674:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.643683:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1d9d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.644252:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23840us (24091us total) trans 4294972269 rc 0/0 00000100:00100000:0.0:1789760718.644263:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4981 00000100:00100000:0.0:1789760718.644273:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697041137319 time 697085684492 state 0x2 ref 0 00000100:00100000:3.0:1789760718.651386:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760718.651496:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515630976 00000100:00100000:1.0:1789760718.651506:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515630976 00000100:00100000:1.0:1789760718.651556:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697041137319 time 697085684492 state 0x0 ref 1 00000100:00100000:1.0:1789760718.651571:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697085944908 time 697085684492 state 0x1 ref 1 00000100:00100000:1.0:1789760718.651603:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697085944908 time 697116891322 state 0x0 ref 1 00000100:00100000:1.0:1789760718.651615:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760718.651660:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20287488, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:1.0:1789760718.651669:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.651690:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.651694:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.651701:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.651753:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.674736:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972270 is committed 00002000:00010000:0.0:1789760718.674965:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20287488 -> 20291584 00010000:00010000:0.0:1789760718.674980:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.674993:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.675002:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.675010:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.675015:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.675025:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.675044:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.675050:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1d9d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.675225:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515630976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23607us (23845us total) trans 4294972270 rc 0/0 00000100:00100000:0.0:1789760718.675234:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4982 00000100:00100000:0.0:1789760718.675243:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697085944908 time 697116891322 state 0x2 ref 0 00000100:00100000:3.0:1789760718.677592:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760718.677679:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631104 00000100:00100000:1.0:1789760718.677688:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631104 00000100:00100000:1.0:1789760718.677874:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697085944908 time 697116891322 state 0x0 ref 1 00000100:00100000:1.0:1789760718.677886:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697117151738 time 697116891322 state 0x1 ref 1 00000100:00100000:1.0:1789760718.677938:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697117151738 time 697143226630 state 0x0 ref 1 00000100:00100000:1.0:1789760718.677951:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760718.677992:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20291584, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:1.0:1789760718.678001:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.678015:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.678019:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.678025:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.678045:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.702702:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972271 is committed 00002000:00010000:0.0:1789760718.702916:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20291584 -> 20295680 00010000:00010000:0.0:1789760718.702932:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.702944:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.702953:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.702965:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.702970:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.702980:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.702998:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.703006:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1d9e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.703220:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25267us (25632us total) trans 4294972271 rc 0/0 00000100:00100000:0.0:1789760718.703229:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4983 00000100:00100000:0.0:1789760718.703239:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697117151738 time 697143226630 state 0x2 ref 0 00000100:00100000:1.0:1789760718.707359:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.707474:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631232 00000100:00100000:0.0:1789760718.707485:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631232 00000100:00100000:0.0:1789760718.707533:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697117151738 time 697143226630 state 0x0 ref 1 00000100:00100000:0.0:1789760718.707547:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697143487046 time 697143226630 state 0x1 ref 1 00000100:00100000:0.0:1789760718.707615:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697143487046 time 697172903103 state 0x0 ref 1 00000100:00100000:0.0:1789760718.707632:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cae850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515631232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.707678:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20295680, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.707686:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.707702:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.707732:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.707739:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.707753:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.758666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972272 is committed 00002000:00010000:0.0:1789760718.758871:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20295680 -> 20299776 00010000:00010000:0.0:1789760718.758886:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.758898:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.758908:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.758916:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.758921:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.758932:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.758951:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.758960:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.759049:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cae850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515631232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51416us (51705us total) trans 4294972272 rc 0/0 00000100:00100000:0.0:1789760718.759061:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4984 00000100:00100000:0.0:1789760718.759080:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697143487046 time 697172903103 state 0x2 ref 0 00000100:00100000:3.0:1789760718.766446:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.766942:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631360 00000100:00100000:0.0:1789760718.766952:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631360 00000100:00100000:0.0:1789760718.767005:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697143487046 time 697172903103 state 0x0 ref 1 00000100:00100000:0.0:1789760718.767020:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697173163519 time 697172903103 state 0x1 ref 1 00000100:00100000:0.0:1789760718.767053:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697173163519 time 697232341816 state 0x0 ref 1 00000100:00100000:0.0:1789760718.767064:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.767111:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20299776, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.767120:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.767135:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.767139:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.767145:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.767169:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.811920:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972273 is committed 00002000:00010000:0.0:1789760718.812081:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20299776 -> 20303872 00010000:00010000:0.0:1789760718.812096:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.812108:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.812117:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.812124:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.812129:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.812139:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.812165:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.812174:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.812262:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45194us (45821us total) trans 4294972273 rc 0/0 00000100:00100000:0.0:1789760718.812272:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4985 00000100:00100000:0.0:1789760718.812281:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697173163519 time 697232341816 state 0x2 ref 0 00000100:00100000:3.0:1789760718.831870:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760718.831973:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631488 00000100:00100000:0.0:1789760718.831983:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631488 00000100:00100000:0.0:1789760718.832036:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697173163519 time 697232341816 state 0x0 ref 1 00000100:00100000:0.0:1789760718.832051:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697232602232 time 697232341816 state 0x1 ref 1 00000100:00100000:0.0:1789760718.832087:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697232602232 time 697297376020 state 0x0 ref 1 00000100:00100000:0.0:1789760718.832100:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760718.832149:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20303872, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xa007 00010000:00010000:0.0:1789760718.832169:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.832183:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.832188:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.832195:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.832214:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.869502:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972274 is committed 00002000:00010000:0.0:1789760718.869614:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20303872 -> 20307968 00002000:00010000:0.0:1789760718.869619:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760718 -> 1789760719 00002000:00010000:0.0:1789760718.869621:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760718 -> 1789760719 00010000:00010000:0.0:1789760718.869633:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.869645:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.869654:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.869662:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760718.869666:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.869676:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.869696:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.869739:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1d9f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.869827:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37724us (37962us total) trans 4294972274 rc 0/0 00000100:00100000:0.0:1789760718.869837:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4986 00000100:00100000:0.0:1789760718.869847:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697232602232 time 697297376020 state 0x2 ref 0 00000100:00100000:2.0:1789760718.876284:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760718.876332:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631616 00000100:00100000:3.0:1789760718.876343:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631616 00000100:00100000:3.0:1789760718.876400:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697232602232 time 697297376020 state 0x0 ref 1 00000100:00100000:3.0:1789760718.876416:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697297636436 time 697297376020 state 0x1 ref 1 00000100:00100000:3.0:1789760718.876451:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697297636436 time 697341739906 state 0x0 ref 1 00000100:00100000:3.0:1789760718.876465:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760718.876510:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20307968, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:3.0:1789760718.876519:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.876534:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760718.876539:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.876545:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.876559:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.925350:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20307968 -> 20312064 00010000:00010000:3.0:1789760718.925371:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.925383:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.925392:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.925400:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760718.925406:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.925416:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.925437:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.925446:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1d9fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.925549:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515631616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49081us (49270us total) trans 4294972275 rc 0/0 00000100:00100000:3.0:1789760718.925560:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4987 00000100:00100000:3.0:1789760718.925569:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697297636436 time 697341739906 state 0x2 ref 0 00000001:00080000:2.0:1789760718.926549:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972275 is committed 00000100:00100000:2.0:1789760718.929194:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760718.929557:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631744 00000100:00100000:1.0:1789760718.929567:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631744 00000100:00100000:1.0:1789760718.929627:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697297636436 time 697341739906 state 0x0 ref 1 00000100:00100000:1.0:1789760718.929643:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697342000322 time 697341739906 state 0x1 ref 1 00000100:00100000:1.0:1789760718.929678:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697342000322 time 697394967112 state 0x0 ref 1 00000100:00100000:1.0:1789760718.929689:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760718.929775:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20312064, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:1.0:1789760718.929784:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.929809:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.929813:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.929819:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.929833:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760719.016734:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20312064 -> 20316160 00010000:00010000:2.0:1789760719.016754:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.016769:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.016778:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.016787:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760719.016802:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.016813:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.016832:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.016841:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1da03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.016930:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515631744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 87238us (87740us total) trans 4294972276 rc 0/0 00000100:00100000:2.0:1789760719.016941:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4988 00000100:00100000:2.0:1789760719.016951:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697342000322 time 697394967112 state 0x2 ref 0 00000001:00080000:3.0:1789760719.020422:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972276 is committed 00000100:00100000:1.0:1789760719.020645:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760719.020746:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515631872 00000100:00100000:0.0:1789760719.020756:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515631872 00000100:00100000:0.0:1789760719.020904:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697342000322 time 697394967112 state 0x0 ref 1 00000100:00100000:0.0:1789760719.020919:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697395227528 time 697394967112 state 0x1 ref 1 00000100:00100000:0.0:1789760719.020953:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697395227528 time 697486241746 state 0x0 ref 1 00000100:00100000:0.0:1789760719.020966:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760719.021016:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20316160, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:0.0:1789760719.021026:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.021041:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760719.021046:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.021052:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.021181:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.083326:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972277 is committed 00002000:00010000:3.0:1789760719.083421:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20316160 -> 20320256 00010000:00010000:3.0:1789760719.083435:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.083506:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.083516:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.083523:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760719.083528:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.083537:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.083555:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.083568:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1da0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.083654:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515631872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62686us (63009us total) trans 4294972277 rc 0/0 00000100:00100000:3.0:1789760719.083663:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4989 00000100:00100000:3.0:1789760719.083672:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697395227528 time 697486241746 state 0x2 ref 0 00000100:00100000:0.0:1789760719.099913:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760719.100014:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632000 00000100:00100000:2.0:1789760719.100024:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632000 00000100:00100000:2.0:1789760719.100159:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697395227528 time 697486241746 state 0x0 ref 1 00000100:00100000:2.0:1789760719.100175:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697486502162 time 697486241746 state 0x1 ref 1 00000100:00100000:2.0:1789760719.100251:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697486502162 time 697565539471 state 0x0 ref 1 00000100:00100000:2.0:1789760719.100263:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515632000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760719.100313:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20320256, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:2.0:1789760719.100322:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.100336:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760719.100341:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.100347:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.100361:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.166126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972278 is committed 00010000:00010000:2.0:1789760719.166212:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760719.166256:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20320256 -> 20324352 00010000:00010000:2.0:1789760719.166270:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.166286:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.166294:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.166301:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760719.166305:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.166313:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.166330:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.166337:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.166433:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515632000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66167us (66526us total) trans 4294972278 rc 0/0 00000100:00100000:2.0:1789760719.166444:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4990 00000100:00100000:2.0:1789760719.166453:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697486502162 time 697565539471 state 0x2 ref 0 00000100:00100000:2.0:1789760719.174033:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760719.176937:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632128 00000100:00100000:3.0:1789760719.176953:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632128 00000100:00100000:3.0:1789760719.177018:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697486502162 time 697565539471 state 0x0 ref 1 00000100:00100000:3.0:1789760719.177034:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697565799887 time 697565539471 state 0x1 ref 1 00000100:00100000:3.0:1789760719.177116:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697565799887 time 697642404881 state 0x0 ref 1 00000100:00100000:3.0:1789760719.177129:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515632128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760719.177209:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20324352, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:3.0:1789760719.177218:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.177234:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760719.177238:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.177244:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.177263:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.214521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972279 is committed 00002000:00010000:3.0:1789760719.214585:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20324352 -> 20328448 00010000:00010000:3.0:1789760719.214601:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.214613:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.214623:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.214632:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760719.214637:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.214647:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.214667:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.214675:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.214791:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515632128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37659us (40763us total) trans 4294972279 rc 0/0 00000100:00100000:3.0:1789760719.214802:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4991 00000100:00100000:3.0:1789760719.214812:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697565799887 time 697642404881 state 0x2 ref 0 00000100:00100000:3.0:1789760719.221350:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760719.221668:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632256 00000100:00100000:2.0:1789760719.221678:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632256 00000100:00100000:2.0:1789760719.221763:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697565799887 time 697642404881 state 0x0 ref 1 00000100:00100000:2.0:1789760719.221778:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697642665297 time 697642404881 state 0x1 ref 1 00000100:00100000:2.0:1789760719.221810:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697642665297 time 697687098870 state 0x0 ref 1 00000100:00100000:2.0:1789760719.221823:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760719.221869:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20328448, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:2.0:1789760719.221878:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.221893:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760719.221898:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.221904:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.221916:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.248475:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972280 is committed 00002000:00010000:3.0:1789760719.248623:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20328448 -> 20332544 00010000:00010000:3.0:1789760719.248639:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.248652:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.248662:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.248670:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760719.248676:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.248686:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.248724:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.248739:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1da1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.248829:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27003us (27483us total) trans 4294972280 rc 0/0 00000100:00100000:3.0:1789760719.248839:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4992 00000100:00100000:3.0:1789760719.248849:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697642665297 time 697687098870 state 0x2 ref 0 00000100:00100000:2.0:1789760719.259341:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760719.259450:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632384 00000100:00100000:3.0:1789760719.259460:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632384 00000100:00100000:3.0:1789760719.259512:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697642665297 time 697687098870 state 0x0 ref 1 00000100:00100000:3.0:1789760719.259527:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697687359286 time 697687098870 state 0x1 ref 1 00000100:00100000:3.0:1789760719.259557:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697687359286 time 697724846006 state 0x0 ref 1 00000100:00100000:3.0:1789760719.259568:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760719.259616:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20332544, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:3.0:1789760719.259625:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.259640:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760719.259644:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.259652:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.259671:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.311767:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972281 is committed 00002000:00010000:1.0:1789760719.311871:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20332544 -> 20336640 00010000:00010000:1.0:1789760719.311886:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.311899:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.311907:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.311914:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760719.311919:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.311928:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.311948:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.311955:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1da26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.312061:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52488us (52724us total) trans 4294972281 rc 0/0 00000100:00100000:1.0:1789760719.312081:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4993 00000100:00100000:1.0:1789760719.312091:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697687359286 time 697724846006 state 0x2 ref 0 00000100:00100000:1.0:1789760719.317970:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760719.318231:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632512 00000100:00100000:2.0:1789760719.318242:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632512 00000100:00100000:2.0:1789760719.318296:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697687359286 time 697724846006 state 0x0 ref 1 00000100:00100000:2.0:1789760719.318311:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697725106422 time 697724846006 state 0x1 ref 1 00000100:00100000:2.0:1789760719.318346:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697725106422 time 697783634813 state 0x0 ref 1 00000100:00100000:2.0:1789760719.318359:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760719.318405:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20336640, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:2.0:1789760719.318414:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.318428:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760719.318432:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.318439:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.318453:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.366248:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972282 is committed 00002000:00010000:2.0:1789760719.366325:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20336640 -> 20340736 00010000:00010000:2.0:1789760719.366340:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.366353:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.366363:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.366377:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760719.366382:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.366392:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.366411:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.366419:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1da2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.366496:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48135us (48533us total) trans 4294972282 rc 0/0 00000100:00100000:2.0:1789760719.366507:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4994 00000100:00100000:2.0:1789760719.366517:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697725106422 time 697783634813 state 0x2 ref 0 00000100:00100000:3.0:1789760719.376828:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760719.383141:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632640 00000100:00100000:0.0:1789760719.383168:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632640 00000100:00100000:0.0:1789760719.383232:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697725106422 time 697783634813 state 0x0 ref 1 00000100:00100000:0.0:1789760719.383247:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697783895229 time 697783634813 state 0x1 ref 1 00000100:00100000:0.0:1789760719.383284:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697783895229 time 697848573196 state 0x0 ref 1 00000100:00100000:0.0:1789760719.383298:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760719.383348:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20340736, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:0.0:1789760719.383357:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.383373:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760719.383378:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.383385:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.383398:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.482949:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972283 is committed 00002000:00010000:0.0:1789760719.483052:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20340736 -> 20344832 00010000:00010000:0.0:1789760719.483137:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.483161:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.483172:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.483179:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760719.483184:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.483194:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.483214:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.483222:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1da34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.483318:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 100017us (106495us total) trans 4294972283 rc 0/0 00000100:00100000:0.0:1789760719.483333:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4995 00000100:00100000:0.0:1789760719.483343:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697783895229 time 697848573196 state 0x2 ref 0 00000100:00100000:2.0:1789760719.487656:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760719.487798:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632768 00000100:00100000:0.0:1789760719.487816:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632768 00000100:00100000:0.0:1789760719.487883:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697783895229 time 697848573196 state 0x0 ref 1 00000100:00100000:0.0:1789760719.487901:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697848833612 time 697848573196 state 0x1 ref 1 00000100:00100000:0.0:1789760719.487925:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697848833612 time 697953214702 state 0x0 ref 1 00000100:00100000:0.0:1789760719.487939:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760719.487989:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20344832, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:0.0:1789760719.487998:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.488014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760719.488019:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.488025:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.488040:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.569495:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972284 is committed 00002000:00010000:0.0:1789760719.569567:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20344832 -> 20348928 00010000:00010000:0.0:1789760719.569582:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.569595:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.569604:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.569612:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760719.569616:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.569624:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.569645:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.569652:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1da3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.569902:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 81960us (82255us total) trans 4294972284 rc 0/0 00000100:00100000:1.0:1789760719.569913:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4996 00000100:00100000:1.0:1789760719.569922:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697848833612 time 697953214702 state 0x2 ref 0 00000100:00100000:1.0:1789760719.574003:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760719.574421:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515632896 00000100:00100000:0.0:1789760719.574435:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515632896 00000100:00100000:0.0:1789760719.574515:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697848833612 time 697953214702 state 0x0 ref 1 00000100:00100000:0.0:1789760719.574534:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697953475118 time 697953214702 state 0x1 ref 1 00000100:00100000:0.0:1789760719.574565:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697953475118 time 698039854451 state 0x0 ref 1 00000100:00100000:0.0:1789760719.574581:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515632896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760719.574643:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20348928, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:0.0:1789760719.574663:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.574684:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760719.574689:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.574696:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.574756:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760719.622364:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20348928 -> 20353024 00010000:00010000:0.0:1789760719.622391:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.622405:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.622415:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.622422:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760719.622428:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.622439:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.622537:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.622546:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1da42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.622675:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515632896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48089us (48679us total) trans 4294972285 rc 0/0 00000100:00100000:0.0:1789760719.622689:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4997 00000100:00100000:0.0:1789760719.622702:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697953475118 time 698039854451 state 0x2 ref 0 00000001:00080000:1.0:1789760719.623272:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972285 is committed 00000100:00100000:0.0:1789760719.626397:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760719.626576:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633024 00000100:00100000:3.0:1789760719.626590:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633024 00000100:00100000:3.0:1789760719.626669:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 697953475118 time 698039854451 state 0x0 ref 1 00000100:00100000:3.0:1789760719.626688:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698040114867 time 698039854451 state 0x1 ref 1 00000100:00100000:3.0:1789760719.626748:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698040114867 time 698092036463 state 0x0 ref 1 00000100:00100000:3.0:1789760719.626763:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760719.626894:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20353024, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:3.0:1789760719.626907:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.626927:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760719.626932:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.626941:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.626960:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.683405:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972286 is committed 00002000:00010000:0.0:1789760719.683542:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20353024 -> 20357120 00010000:00010000:0.0:1789760719.683564:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.683580:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.683591:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.683606:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760719.683612:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.683623:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.683649:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.683658:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1da49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.683829:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57061us (57441us total) trans 4294972286 rc 0/0 00000100:00100000:0.0:1789760719.683841:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4998 00000100:00100000:0.0:1789760719.683853:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698040114867 time 698092036463 state 0x2 ref 0 00000100:00100000:2.0:1789760719.687287:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760719.687398:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633152 00000100:00100000:1.0:1789760719.687411:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633152 00000100:00100000:1.0:1789760719.687545:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698040114867 time 698092036463 state 0x0 ref 1 00000100:00100000:1.0:1789760719.687564:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698092296879 time 698092036463 state 0x1 ref 1 00000100:00100000:1.0:1789760719.687643:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698092296879 time 698152931319 state 0x0 ref 1 00000100:00100000:1.0:1789760719.687659:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760719.687801:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20357120, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:1.0:1789760719.687815:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.687834:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.687840:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.687848:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.687864:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.745582:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972287 is committed 00002000:00010000:1.0:1789760719.745668:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20357120 -> 20361216 00010000:00010000:1.0:1789760719.745688:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.745703:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.745735:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.745744:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760719.745750:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.745760:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.745852:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.745863:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1da50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.746121:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58457us (58840us total) trans 4294972287 rc 0/0 00000100:00100000:1.0:1789760719.746134:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 4999 00000100:00100000:1.0:1789760719.746157:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698092296879 time 698152931319 state 0x2 ref 0 00000100:00100000:0.0:1789760719.750189:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760719.750344:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633280 00000100:00100000:3.0:1789760719.750357:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633280 00000100:00100000:3.0:1789760719.750427:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698092296879 time 698152931319 state 0x0 ref 1 00000100:00100000:3.0:1789760719.750446:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698153191735 time 698152931319 state 0x1 ref 1 00000100:00100000:3.0:1789760719.750485:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698153191735 time 698215773792 state 0x0 ref 1 00000100:00100000:3.0:1789760719.750498:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515633280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760719.750566:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20361216, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:3.0:1789760719.750578:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.750598:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760719.750604:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.750613:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.750631:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.802559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972288 is committed 00002000:00010000:3.0:1789760719.802770:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20361216 -> 20365312 00010000:00010000:3.0:1789760719.802790:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.802813:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.802823:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.802831:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760719.802838:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.802849:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.802874:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.802883:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.802997:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515633280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52493us (52815us total) trans 4294972288 rc 0/0 00000100:00100000:3.0:1789760719.803009:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5000 00000100:00100000:3.0:1789760719.803023:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698153191735 time 698215773792 state 0x2 ref 0 00000100:00100000:1.0:1789760719.809765:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760719.809843:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633408 00000100:00100000:0.0:1789760719.809863:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633408 00000100:00100000:0.0:1789760719.809959:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698153191735 time 698215773792 state 0x0 ref 1 00000100:00100000:0.0:1789760719.809980:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698216034208 time 698215773792 state 0x1 ref 1 00000100:00100000:0.0:1789760719.810026:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698216034208 time 698275314505 state 0x0 ref 1 00000100:00100000:0.0:1789760719.810042:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760719.810109:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20365312, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:0.0:1789760719.810120:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.810162:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760719.810168:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.810176:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.810191:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.896418:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972289 is committed 00002000:00010000:1.0:1789760719.896550:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20365312 -> 20369408 00010000:00010000:1.0:1789760719.896565:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.896579:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.896588:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.896596:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760719.896602:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.896611:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.896631:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.896638:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1da5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.896761:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 86717us (87002us total) trans 4294972289 rc 0/0 00000100:00100000:1.0:1789760719.896772:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5001 00000100:00100000:1.0:1789760719.896782:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698216034208 time 698275314505 state 0x2 ref 0 00000100:00100000:1.0:1789760719.906210:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760719.906310:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633536 00000100:00100000:2.0:1789760719.906321:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633536 00000100:00100000:2.0:1789760719.906383:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698216034208 time 698275314505 state 0x0 ref 1 00000100:00100000:2.0:1789760719.906398:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698275574921 time 698275314505 state 0x1 ref 1 00000100:00100000:2.0:1789760719.906432:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698275574921 time 698371720729 state 0x0 ref 1 00000100:00100000:2.0:1789760719.906445:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760719.906506:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20369408, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xa007 00010000:00010000:2.0:1789760719.906515:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.906529:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760719.906533:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.906540:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.906554:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.964497:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972290 is committed 00002000:00010000:2.0:1789760719.964626:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20369408 -> 20373504 00002000:00010000:2.0:1789760719.964631:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760719 -> 1789760720 00002000:00010000:2.0:1789760719.964633:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760719 -> 1789760720 00010000:00010000:2.0:1789760719.964646:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.964659:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.964668:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.964676:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760719.964681:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.964690:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.964943:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.964953:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1da65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.965206:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515633536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58759us (59002us total) trans 4294972290 rc 0/0 00000100:00100000:2.0:1789760719.965217:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5002 00000100:00100000:2.0:1789760719.965226:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698275574921 time 698371720729 state 0x2 ref 0 00000100:00100000:0.0:1789760719.972684:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760719.973033:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633664 00000100:00100000:1.0:1789760719.973043:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633664 00000100:00100000:0.0:1789760719.973239:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698275574921 time 698371720729 state 0x0 ref 1 00000100:00100000:0.0:1789760719.973254:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698371981145 time 698371720729 state 0x1 ref 1 00000100:00100000:0.0:1789760719.973319:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698371981145 time 698438607878 state 0x0 ref 1 00000100:00100000:0.0:1789760719.973333:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515633664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760719.973391:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20373504, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760719.973401:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.973418:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760719.973423:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.973429:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.973443:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.012228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972291 is committed 00002000:00010000:1.0:1789760720.012372:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20373504 -> 20377600 00010000:00010000:1.0:1789760720.012388:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.012401:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.012411:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.012424:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760720.012429:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.012438:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.012459:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.012467:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.012626:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515633664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39289us (39945us total) trans 4294972291 rc 0/0 00000100:00100000:1.0:1789760720.012637:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5003 00000100:00100000:1.0:1789760720.012646:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698371981145 time 698438607878 state 0x2 ref 0 00000100:00100000:1.0:1789760720.017086:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760720.017354:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633792 00000100:00100000:2.0:1789760720.017365:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633792 00000100:00100000:2.0:1789760720.017471:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698371981145 time 698438607878 state 0x0 ref 1 00000100:00100000:2.0:1789760720.017486:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698438868294 time 698438607878 state 0x1 ref 1 00000100:00100000:2.0:1789760720.017514:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698438868294 time 698482803773 state 0x0 ref 1 00000100:00100000:2.0:1789760720.017530:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515633792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760720.017584:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20377600, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:2.0:1789760720.017592:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.017608:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760720.017612:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.017618:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.017631:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760720.021701:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760720.073626:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972292 is committed 00002000:00010000:0.0:1789760720.073772:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20377600 -> 20381696 00010000:00010000:0.0:1789760720.073787:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.073800:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.073810:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.073825:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.073830:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.073840:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.073865:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.073873:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1da73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.073974:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515633792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56441us (56893us total) trans 4294972292 rc 0/0 00000100:00100000:0.0:1789760720.073984:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5004 00000100:00100000:0.0:1789760720.073994:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698438868294 time 698482803773 state 0x2 ref 0 00000100:00100000:2.0:1789760720.078808:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760720.078896:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515633920 00000100:00100000:1.0:1789760720.078916:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515633920 00000100:00100000:1.0:1789760720.078972:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698438868294 time 698482803773 state 0x0 ref 1 00000100:00100000:1.0:1789760720.078987:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698483064189 time 698482803773 state 0x1 ref 1 00000100:00100000:1.0:1789760720.079016:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698483064189 time 698544305656 state 0x0 ref 1 00000100:00100000:1.0:1789760720.079029:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515633920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760720.079076:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20381696, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:1.0:1789760720.079085:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.079100:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760720.079105:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.079111:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.079125:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760720.130424:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20381696 -> 20385792 00010000:00010000:3.0:1789760720.130444:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.130458:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.130466:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.130474:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760720.130479:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.130498:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.130518:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.130526:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1da7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.130629:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515633920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51597us (51826us total) trans 4294972293 rc 0/0 00000100:00100000:3.0:1789760720.130640:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5005 00000100:00100000:3.0:1789760720.130649:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698483064189 time 698544305656 state 0x2 ref 0 00000001:00080000:2.0:1789760720.131347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972293 is committed 00000100:00100000:1.0:1789760720.134229:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760720.134385:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634048 00000100:00100000:3.0:1789760720.134396:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634048 00000100:00100000:3.0:1789760720.134445:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698483064189 time 698544305656 state 0x0 ref 1 00000100:00100000:3.0:1789760720.134460:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698544566072 time 698544305656 state 0x1 ref 1 00000100:00100000:3.0:1789760720.134494:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698544566072 time 698599776207 state 0x0 ref 1 00000100:00100000:3.0:1789760720.134506:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515634048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760720.134557:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20385792, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:3.0:1789760720.134566:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.134581:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760720.134585:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.134591:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.134605:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.190318:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760720.200480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972294 is committed 00002000:00010000:3.0:1789760720.200528:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20385792 -> 20389888 00010000:00010000:3.0:1789760720.200543:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.200561:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.200569:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.200576:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760720.200581:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.200591:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.200611:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.200619:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1da81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.200930:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515634048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66421us (66705us total) trans 4294972294 rc 0/0 00000100:00100000:3.0:1789760720.200941:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5006 00000100:00100000:3.0:1789760720.200950:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698544566072 time 698599776207 state 0x2 ref 0 00000100:00100000:1.0:1789760720.203967:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760720.204077:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634176 00000100:00100000:0.0:1789760720.204087:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634176 00000100:00100000:0.0:1789760720.204136:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698544566072 time 698599776207 state 0x0 ref 1 00000100:00100000:0.0:1789760720.204162:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698600036623 time 698599776207 state 0x1 ref 1 00000100:00100000:0.0:1789760720.204191:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698600036623 time 698669480509 state 0x0 ref 1 00000100:00100000:0.0:1789760720.204202:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760720.204246:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20389888, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760720.204254:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.204270:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760720.204275:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.204281:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.204296:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.268001:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972295 is committed 00002000:00010000:0.0:1789760720.268202:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20389888 -> 20393984 00010000:00010000:0.0:1789760720.268216:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.268229:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.268238:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.268245:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.268250:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.268259:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.268276:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.268282:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1da88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.268369:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64163us (64406us total) trans 4294972295 rc 0/0 00000100:00100000:0.0:1789760720.268380:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5007 00000100:00100000:0.0:1789760720.268389:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698600036623 time 698669480509 state 0x2 ref 0 00000100:00100000:2.0:1789760720.272296:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760720.272478:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634304 00000100:00100000:1.0:1789760720.272488:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634304 00000100:00100000:1.0:1789760720.272545:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698600036623 time 698669480509 state 0x0 ref 1 00000100:00100000:1.0:1789760720.272562:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698669740925 time 698669480509 state 0x1 ref 1 00000100:00100000:1.0:1789760720.272592:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698669740925 time 698737881423 state 0x0 ref 1 00000100:00100000:1.0:1789760720.272608:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515634304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760720.272658:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20393984, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:1.0:1789760720.272667:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.272682:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760720.272686:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.272692:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.272738:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760720.341521:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20393984 -> 20398080 00010000:00010000:2.0:1789760720.341540:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.341553:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.341562:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.341574:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760720.341579:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.341587:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.341605:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.341613:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1da8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.341740:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515634304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69103us (69421us total) trans 4294972296 rc 0/0 00000100:00100000:2.0:1789760720.341751:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5008 00000100:00100000:2.0:1789760720.341759:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698669740925 time 698737881423 state 0x2 ref 0 00000001:00080000:3.0:1789760720.342022:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972296 is committed 00000100:00100000:3.0:1789760720.345733:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760720.345944:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634432 00000100:00100000:0.0:1789760720.345955:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634432 00000100:00100000:0.0:1789760720.346007:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698669740925 time 698737881423 state 0x0 ref 1 00000100:00100000:0.0:1789760720.346023:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698738141839 time 698737881423 state 0x1 ref 1 00000100:00100000:0.0:1789760720.346053:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698738141839 time 698811342735 state 0x0 ref 1 00000100:00100000:0.0:1789760720.346073:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515634432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760720.346122:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20398080, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760720.346131:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.346156:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760720.346161:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.346167:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.346179:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760720.410470:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20398080 -> 20402176 00010000:00010000:0.0:1789760720.410490:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.410503:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.410511:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.410518:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.410522:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.410530:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.410549:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.410556:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1da96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.410645:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515634432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64570us (64955us total) trans 4294972297 rc 0/0 00000100:00100000:0.0:1789760720.410766:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5009 00000100:00100000:0.0:1789760720.410774:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698738141839 time 698811342735 state 0x2 ref 0 00000001:00080000:3.0:1789760720.411438:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972297 is committed 00000100:00100000:0.0:1789760720.420293:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760720.420414:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634560 00000100:00100000:2.0:1789760720.420427:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634560 00000100:00100000:2.0:1789760720.420497:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698738141839 time 698811342735 state 0x0 ref 1 00000100:00100000:2.0:1789760720.420517:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698811603151 time 698811342735 state 0x1 ref 1 00000100:00100000:2.0:1789760720.420547:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698811603151 time 698885836486 state 0x0 ref 1 00000100:00100000:2.0:1789760720.420565:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760720.420633:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20402176, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:2.0:1789760720.420643:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.420662:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760720.420668:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.420675:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.420691:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.456264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972298 is committed 00002000:00010000:0.0:1789760720.456394:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20402176 -> 20406272 00010000:00010000:0.0:1789760720.456416:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.456432:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.456442:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.456450:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.456456:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.456467:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.456492:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.456499:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1da9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.456665:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36094us (36378us total) trans 4294972298 rc 0/0 00000100:00100000:0.0:1789760720.456677:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5010 00000100:00100000:0.0:1789760720.456688:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698811603151 time 698885836486 state 0x2 ref 0 00000100:00100000:3.0:1789760720.459684:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760720.459840:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634688 00000100:00100000:1.0:1789760720.459853:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634688 00000100:00100000:1.0:1789760720.459923:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698811603151 time 698885836486 state 0x0 ref 1 00000100:00100000:1.0:1789760720.459942:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698886096902 time 698885836486 state 0x1 ref 1 00000100:00100000:1.0:1789760720.459979:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698886096902 time 698925267636 state 0x0 ref 1 00000100:00100000:1.0:1789760720.459992:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515634688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760720.460055:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20406272, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:1.0:1789760720.460075:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.460100:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760720.460105:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.460112:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.460129:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.492938:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972299 is committed 00002000:00010000:1.0:1789760720.493378:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20406272 -> 20410368 00010000:00010000:1.0:1789760720.493397:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.493414:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.493424:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.493432:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760720.493438:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.493447:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.493472:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.493481:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1daa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.493632:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515634688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33634us (33954us total) trans 4294972299 rc 0/0 00000100:00100000:1.0:1789760720.493643:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5011 00000100:00100000:1.0:1789760720.493654:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698886096902 time 698925267636 state 0x2 ref 0 00000100:00100000:2.0:1789760720.497261:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760720.497361:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634816 00000100:00100000:3.0:1789760720.497374:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634816 00000100:00100000:3.0:1789760720.497440:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698886096902 time 698925267636 state 0x0 ref 1 00000100:00100000:3.0:1789760720.497457:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698925528052 time 698925267636 state 0x1 ref 1 00000100:00100000:3.0:1789760720.497582:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698925528052 time 698962869353 state 0x0 ref 1 00000100:00100000:3.0:1789760720.497596:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760720.497655:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20410368, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:3.0:1789760720.497665:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.497684:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760720.497689:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.497696:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.497744:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.523325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972300 is committed 00002000:00010000:3.0:1789760720.523608:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20410368 -> 20414464 00010000:00010000:3.0:1789760720.523628:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.523643:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.523653:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.523661:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760720.523667:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.523676:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.523701:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.523845:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1daab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.524015:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26413us (26759us total) trans 4294972300 rc 0/0 00000100:00100000:3.0:1789760720.524028:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5012 00000100:00100000:3.0:1789760720.524041:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698925528052 time 698962869353 state 0x2 ref 0 00000100:00100000:1.0:1789760720.527501:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760720.527603:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515634944 00000100:00100000:0.0:1789760720.527616:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515634944 00000100:00100000:0.0:1789760720.527688:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698925528052 time 698962869353 state 0x0 ref 1 00000100:00100000:0.0:1789760720.527734:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698963129769 time 698962869353 state 0x1 ref 1 00000100:00100000:0.0:1789760720.527774:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698963129769 time 698993062511 state 0x0 ref 1 00000100:00100000:0.0:1789760720.527930:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760720.527996:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20414464, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760720.528007:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.528026:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760720.528031:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.528039:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.528055:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.559685:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972301 is committed 00002000:00010000:3.0:1789760720.559850:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20414464 -> 20418560 00010000:00010000:3.0:1789760720.559870:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.559886:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.559896:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.559904:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760720.559909:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.559919:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.559945:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.559953:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dab2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.560075:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515634944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32282us (32581us total) trans 4294972301 rc 0/0 00000100:00100000:3.0:1789760720.560087:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5013 00000100:00100000:3.0:1789760720.560101:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698963129769 time 698993062511 state 0x2 ref 0 00000100:00100000:3.0:1789760720.562918:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760720.563261:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635072 00000100:00100000:2.0:1789760720.563275:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635072 00000100:00100000:2.0:1789760720.563345:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698963129769 time 698993062511 state 0x0 ref 1 00000100:00100000:2.0:1789760720.563364:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698993322927 time 698993062511 state 0x1 ref 1 00000100:00100000:2.0:1789760720.563401:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698993322927 time 699028690220 state 0x0 ref 1 00000100:00100000:2.0:1789760720.563416:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515635072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760720.563482:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20418560, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:2.0:1789760720.563494:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.563515:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760720.563521:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.563528:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.563545:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760720.594536:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20418560 -> 20422656 00010000:00010000:2.0:1789760720.594562:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.594577:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.594588:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.594596:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760720.594602:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.594612:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.594641:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.594649:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dab9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.594825:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515635072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31406us (31916us total) trans 4294972302 rc 0/0 00000100:00100000:2.0:1789760720.594838:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5014 00000100:00100000:2.0:1789760720.594849:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698993322927 time 699028690220 state 0x2 ref 0 00000001:00080000:3.0:1789760720.594990:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972302 is committed 00000100:00100000:2.0:1789760720.598480:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760720.598583:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635200 00000100:00100000:0.0:1789760720.598596:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635200 00000100:00100000:0.0:1789760720.598668:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 698993322927 time 699028690220 state 0x0 ref 1 00000100:00100000:0.0:1789760720.598687:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699028950636 time 699028690220 state 0x1 ref 1 00000100:00100000:0.0:1789760720.599903:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699028950636 time 699065181704 state 0x0 ref 1 00000100:00100000:0.0:1789760720.599932:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760720.599996:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20422656, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760720.600007:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.600027:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760720.600032:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.600039:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.600056:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.625926:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972303 is committed 00002000:00010000:0.0:1789760720.626318:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20422656 -> 20426752 00010000:00010000:0.0:1789760720.626339:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.626361:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.626371:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.626379:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.626384:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.626395:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.626419:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.626426:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dac0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.626545:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26618us (28072us total) trans 4294972303 rc 0/0 00000100:00100000:0.0:1789760720.626558:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5015 00000100:00100000:0.0:1789760720.626570:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699028950636 time 699065181704 state 0x2 ref 0 00000100:00100000:1.0:1789760720.629934:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760720.630035:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635328 00000100:00100000:0.0:1789760720.630050:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635328 00000100:00100000:0.0:1789760720.630123:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699028950636 time 699065181704 state 0x0 ref 1 00000100:00100000:0.0:1789760720.630151:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699065442120 time 699065181704 state 0x1 ref 1 00000100:00100000:0.0:1789760720.630186:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699065442120 time 699095474271 state 0x0 ref 1 00000100:00100000:0.0:1789760720.630201:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760720.630267:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20426752, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760720.630278:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.630299:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760720.630305:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.630312:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.630330:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.698972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972304 is committed 00002000:00010000:0.0:1789760720.704418:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20426752 -> 20430848 00010000:00010000:0.0:1789760720.704445:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.704460:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.704477:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.704485:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.704492:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.704503:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.704528:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.704536:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dac7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.704675:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74470us (74748us total) trans 4294972304 rc 0/0 00000100:00100000:0.0:1789760720.704688:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5016 00000100:00100000:0.0:1789760720.704700:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699065442120 time 699095474271 state 0x2 ref 0 00000100:00100000:3.0:1789760720.707662:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760720.708082:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635456 00000100:00100000:1.0:1789760720.708095:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635456 00000100:00100000:1.0:1789760720.708176:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699065442120 time 699095474271 state 0x0 ref 1 00000100:00100000:1.0:1789760720.708195:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699095734687 time 699095474271 state 0x1 ref 1 00000100:00100000:1.0:1789760720.708238:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699095734687 time 699173525879 state 0x0 ref 1 00000100:00100000:1.0:1789760720.708252:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760720.708314:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20430848, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:1.0:1789760720.708325:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708347:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.708352:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.708360:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708378:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.807018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972305 is committed 00002000:00010000:0.0:1789760720.807214:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20430848 -> 20434944 00010000:00010000:0.0:1789760720.807234:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.807256:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.807266:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.807274:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.807280:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.807290:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.807315:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.807323:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dace lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.807678:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 99418us (100021us total) trans 4294972305 rc 0/0 00000100:00100000:0.0:1789760720.807691:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5017 00000100:00100000:0.0:1789760720.807704:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699095734687 time 699173525879 state 0x2 ref 0 00000100:00100000:3.0:1789760720.826047:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760720.826245:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635584 00000100:00100000:0.0:1789760720.826260:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635584 00000100:00100000:0.0:1789760720.826335:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699095734687 time 699173525879 state 0x0 ref 1 00000100:00100000:0.0:1789760720.826355:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699173786295 time 699173525879 state 0x1 ref 1 00000100:00100000:0.0:1789760720.826396:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699173786295 time 699291684018 state 0x0 ref 1 00000100:00100000:0.0:1789760720.826725:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515635584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760720.826808:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20434944, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xa007 00010000:00010000:0.0:1789760720.826819:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.826842:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760720.826848:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.826856:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.826993:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.939453:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972306 is committed 00002000:00010000:0.0:1789760720.939648:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20434944 -> 20439040 00002000:00010000:0.0:1789760720.939653:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760720 -> 1789760721 00002000:00010000:0.0:1789760720.939656:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760720 -> 1789760721 00010000:00010000:0.0:1789760720.939670:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.939683:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.939693:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.939701:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760720.939772:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.939783:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.939803:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.939810:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dad5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.939938:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515635584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 113525us (114064us total) trans 4294972306 rc 0/0 00000100:00100000:0.0:1789760720.939949:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5018 00000100:00100000:0.0:1789760720.939957:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699173786295 time 699291684018 state 0x2 ref 0 00000100:00100000:1.0:1789760720.942923:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760720.943584:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635712 00000100:00100000:3.0:1789760720.943603:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635712 00000100:00100000:3.0:1789760720.943684:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699173786295 time 699291684018 state 0x0 ref 1 00000100:00100000:3.0:1789760720.943966:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699291944434 time 699291684018 state 0x1 ref 1 00000100:00100000:3.0:1789760720.944007:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699291944434 time 699409294970 state 0x0 ref 1 00000100:00100000:3.0:1789760720.944027:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760720.944097:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20439040, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:3.0:1789760720.944107:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.944127:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760720.944132:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.944148:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.944166:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.027367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972307 is committed 00002000:00010000:1.0:1789760721.027463:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20439040 -> 20443136 00010000:00010000:1.0:1789760721.027477:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.027495:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.027504:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.027512:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.027517:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.027526:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.027547:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.027554:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1dadc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.027651:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 83625us (84732us total) trans 4294972307 rc 0/0 00000100:00100000:1.0:1789760721.027663:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5019 00000100:00100000:1.0:1789760721.027673:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699291944434 time 699409294970 state 0x2 ref 0 00000100:00100000:3.0:1789760721.033292:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.034480:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635840 00000100:00100000:2.0:1789760721.034499:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635840 00000100:00100000:2.0:1789760721.034558:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699291944434 time 699409294970 state 0x0 ref 1 00000100:00100000:2.0:1789760721.034574:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699409555386 time 699409294970 state 0x1 ref 1 00000100:00100000:2.0:1789760721.034608:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699409555386 time 699499897462 state 0x0 ref 1 00000100:00100000:2.0:1789760721.034622:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515635840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760721.034673:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20443136, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:2.0:1789760721.034683:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.035155:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760721.035162:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.035169:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.035314:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760721.091762:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20443136 -> 20447232 00010000:00010000:2.0:1789760721.091791:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.091802:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.091808:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.091813:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760721.091816:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.091821:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.091838:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.091843:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dae3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.091928:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515635840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57299us (58746us total) trans 4294972308 rc 0/0 00000100:00100000:2.0:1789760721.091938:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5020 00000100:00100000:2.0:1789760721.091948:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699409555386 time 699499897462 state 0x2 ref 0 00000001:00080000:1.0:1789760721.092089:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972308 is committed 00000100:00100000:3.0:1789760721.101307:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760721.102062:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515635968 00000100:00100000:1.0:1789760721.102084:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515635968 00000100:00100000:1.0:1789760721.102149:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699409555386 time 699499897462 state 0x0 ref 1 00000100:00100000:1.0:1789760721.102165:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699500157878 time 699499897462 state 0x1 ref 1 00000100:00100000:1.0:1789760721.102247:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699500157878 time 699567535330 state 0x0 ref 1 00000100:00100000:1.0:1789760721.102261:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515635968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760721.102311:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20447232, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:1.0:1789760721.102320:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.102335:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760721.102340:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.102346:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.102360:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.136525:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20447232 -> 20451328 00010000:00010000:1.0:1789760721.136544:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.136558:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.136567:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.136574:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.136579:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.136588:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.136607:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.136614:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1daea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.136734:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515635968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34442us (35403us total) trans 4294972309 rc 0/0 00000100:00100000:1.0:1789760721.136750:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5021 00000100:00100000:1.0:1789760721.136759:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699500157878 time 699567535330 state 0x2 ref 0 00000001:00080000:2.0:1789760721.137584:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972309 is committed 00000100:00100000:3.0:1789760721.143446:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760721.143549:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636096 00000100:00100000:0.0:1789760721.143560:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636096 00000100:00100000:0.0:1789760721.143613:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699500157878 time 699567535330 state 0x0 ref 1 00000100:00100000:0.0:1789760721.143629:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699567795746 time 699567535330 state 0x1 ref 1 00000100:00100000:0.0:1789760721.143660:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699567795746 time 699608948900 state 0x0 ref 1 00000100:00100000:0.0:1789760721.143672:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515636096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760721.143757:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20451328, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:0.0:1789760721.143766:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.143795:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760721.143800:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.143806:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.143820:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.195535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972310 is committed 00002000:00010000:1.0:1789760721.195773:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20451328 -> 20455424 00010000:00010000:1.0:1789760721.195873:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195887:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195897:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195906:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.195912:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195922:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195940:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195948:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1daf1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.196048:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515636096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52373us (52606us total) trans 4294972310 rc 0/0 00000100:00100000:1.0:1789760721.197496:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5022 00000100:00100000:1.0:1789760721.197508:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699567795746 time 699608948900 state 0x2 ref 0 00000100:00100000:3.0:1789760721.200441:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760721.200644:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636224 00000100:00100000:0.0:1789760721.200656:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636224 00000100:00100000:0.0:1789760721.200747:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699567795746 time 699608948900 state 0x0 ref 1 00000100:00100000:0.0:1789760721.200763:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699609209316 time 699608948900 state 0x1 ref 1 00000100:00100000:0.0:1789760721.200794:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699609209316 time 699666082741 state 0x0 ref 1 00000100:00100000:0.0:1789760721.200807:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760721.200853:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20455424, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:0.0:1789760721.200863:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.200879:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760721.200884:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.200890:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.200904:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.213827:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760721.247222:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972311 is committed 00002000:00010000:3.0:1789760721.247329:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20455424 -> 20459520 00010000:00010000:3.0:1789760721.247346:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.247360:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.247371:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.247378:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760721.247384:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.247392:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.247412:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.247425:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1daf8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.247518:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46708us (47082us total) trans 4294972311 rc 0/0 00000100:00100000:3.0:1789760721.247528:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5023 00000100:00100000:3.0:1789760721.247538:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699609209316 time 699666082741 state 0x2 ref 0 00000100:00100000:3.0:1789760721.251561:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760721.251648:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636352 00000100:00100000:0.0:1789760721.251659:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636352 00000100:00100000:0.0:1789760721.251753:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699609209316 time 699666082741 state 0x0 ref 1 00000100:00100000:0.0:1789760721.251770:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699666343157 time 699666082741 state 0x1 ref 1 00000100:00100000:0.0:1789760721.251801:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699666343157 time 699717090160 state 0x0 ref 1 00000100:00100000:0.0:1789760721.251815:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515636352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760721.251868:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20459520, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:0.0:1789760721.251877:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.251893:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760721.251898:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.251904:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.251919:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.358908:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972312 is committed 00002000:00010000:3.0:1789760721.359462:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20459520 -> 20463616 00010000:00010000:3.0:1789760721.359479:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.359492:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.359501:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.359508:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760721.359513:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.359522:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.359552:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.359560:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1daff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.359665:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515636352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 107847us (108109us total) trans 4294972312 rc 0/0 00000100:00100000:3.0:1789760721.359676:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5024 00000100:00100000:3.0:1789760721.359686:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699666343157 time 699717090160 state 0x2 ref 0 00000100:00100000:1.0:1789760721.364340:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.364443:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636480 00000100:00100000:2.0:1789760721.364453:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636480 00000100:00100000:2.0:1789760721.364508:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699666343157 time 699717090160 state 0x0 ref 1 00000100:00100000:2.0:1789760721.364523:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699717350576 time 699717090160 state 0x1 ref 1 00000100:00100000:2.0:1789760721.364555:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699717350576 time 699829843895 state 0x0 ref 1 00000100:00100000:2.0:1789760721.364568:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515636480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760721.364630:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20463616, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:2.0:1789760721.364638:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.364654:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760721.364659:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.364666:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.364681:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.414184:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544407936:0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.414252:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760721.414307:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8029c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544408448:0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.414331:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760721.414359:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760721.414365:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544408064:0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.414383:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760721.414413:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407936 00000100:00100000:0.0:1789760721.414420:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407936 00000100:00100000:0.0:1789760721.414455:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408064 00000100:00100000:0.0:1789760721.414457:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408064 00000100:00100000:2.0:1789760721.414682:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544408192:0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.414734:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760721.414762:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760721.414768:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544407808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.414806:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760721.414827:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8029c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544408320:0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.414848:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760721.414875:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408448 00000100:00100000:2.0:1789760721.414879:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408448 00000100:00100000:2.0:1789760721.414906:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408192 00000100:00100000:2.0:1789760721.414909:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408192 00000100:00100000:2.0:1789760721.414929:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1789760721.414937:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544408448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.414988:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544408448:12345-0@lo:400:kworker.0 Request processed in 51us (656us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760721.414995:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1789760721.415010:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789760721.415016:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806191f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544408192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.415061:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806191f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544408192:12345-0@lo:400:kworker.0 Request processed in 45us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760721.415067:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789760721.415084:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544407808 00000100:00100000:2.0:1789760721.415088:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544407808 00000100:00100000:2.0:1789760721.415109:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789760721.415115:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92025f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544407808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.415151:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92025f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544407808:12345-0@lo:400:kworker.0 Request processed in 36us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760721.415156:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789760721.415182:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408320 00000100:00100000:2.0:1789760721.415186:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408320 00000100:00100000:0.0:1789760721.415504:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1789760721.415515:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544407936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.415547:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544407936:12345-0@lo:400:kworker.0 Request processed in 36us (1296us total) trans 0 rc 0/0 00000100:00080000:1.0:1789760721.416929:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760721.416938:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760721.416941:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760721.417320:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:2.0:1789760721.417335:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805525c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544408064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.417404:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805525c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544408064:12345-0@lo:400:kworker.0 Request processed in 76us (3019us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760721.417410:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:2.0:1789760721.417427:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789760721.417433:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806192d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544408320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760721.417460:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806192d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544408320:12345-0@lo:400:kworker.0 Request processed in 28us (2611us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760721.417467:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1789760721.417629:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544407808:0@lo:400:kworker.0 00000100:00100000:3.0:1789760721.417644:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8029c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544408320:0@lo:400:kworker.0 00000100:00100000:3.0:1789760721.417656:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544408192:0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.418614:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1789760721.418684:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544408064:0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.418697:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544407936:0@lo:400:kworker.0 00000100:00100000:0.0:1789760721.418727:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8029c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544408448:0@lo:400:kworker.0 00002000:00010000:1.0:1789760721.425002:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20463616 -> 20467712 00010000:00010000:1.0:1789760721.425020:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.425035:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.425172:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.425183:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.425198:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.425210:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.425235:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.425245:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1db06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.425338:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515636480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60768us (61003us total) trans 4294972313 rc 0/0 00000100:00100000:1.0:1789760721.425348:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5025 00000100:00100000:1.0:1789760721.425358:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699717350576 time 699829843895 state 0x2 ref 0 00000001:00080000:0.0:1789760721.426768:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972313 is committed 00000100:00100000:1.0:1789760721.439846:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.439937:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636608 00000100:00100000:2.0:1789760721.439946:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636608 00000100:00100000:2.0:1789760721.439999:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699717350576 time 699829843895 state 0x0 ref 1 00000100:00100000:2.0:1789760721.440014:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699830104311 time 699829843895 state 0x1 ref 1 00000100:00100000:2.0:1789760721.440051:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699830104311 time 699905333009 state 0x0 ref 1 00000100:00100000:2.0:1789760721.440068:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515636608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760721.440116:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20467712, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:2.0:1789760721.440124:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.440150:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760721.440154:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.440161:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.440177:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.493461:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972314 is committed 00002000:00010000:1.0:1789760721.494295:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20467712 -> 20471808 00010000:00010000:1.0:1789760721.494312:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.494323:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.494332:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.494339:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.494343:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.494357:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.494375:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.494382:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1db0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.494471:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515636608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54401us (54629us total) trans 4294972314 rc 0/0 00000100:00100000:1.0:1789760721.494490:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5026 00000100:00100000:1.0:1789760721.494499:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699830104311 time 699905333009 state 0x2 ref 0 00000100:00100000:3.0:1789760721.502220:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760721.502297:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636736 00000100:00100000:1.0:1789760721.502307:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636736 00000100:00100000:1.0:1789760721.502356:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699830104311 time 699905333009 state 0x0 ref 1 00000100:00100000:1.0:1789760721.502370:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699905593425 time 699905333009 state 0x1 ref 1 00000100:00100000:1.0:1789760721.502396:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699905593425 time 699967686024 state 0x0 ref 1 00000100:00100000:1.0:1789760721.502409:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760721.502451:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20471808, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:1.0:1789760721.502460:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.502477:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760721.502481:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.502488:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.502501:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.603826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972315 is committed 00002000:00010000:1.0:1789760721.604220:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20471808 -> 20475904 00010000:00010000:1.0:1789760721.604237:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.604249:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.604258:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.604264:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.604269:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.604278:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.604299:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.604307:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.604392:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 101980us (102177us total) trans 4294972315 rc 0/0 00000100:00100000:1.0:1789760721.604403:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5027 00000100:00100000:1.0:1789760721.604413:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699905593425 time 699967686024 state 0x2 ref 0 00000100:00100000:3.0:1789760721.611423:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760721.614535:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636864 00000100:00100000:1.0:1789760721.614549:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636864 00000100:00100000:1.0:1789760721.614613:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699905593425 time 699967686024 state 0x0 ref 1 00000100:00100000:1.0:1789760721.614631:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699967946440 time 699967686024 state 0x1 ref 1 00000100:00100000:1.0:1789760721.614673:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699967946440 time 700079962373 state 0x0 ref 1 00000100:00100000:1.0:1789760721.614686:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760721.614778:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20475904, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:1.0:1789760721.614787:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.614803:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.614807:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.614813:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.614831:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.690481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972316 is committed 00002000:00010000:1.0:1789760721.690661:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20475904 -> 20480000 00010000:00010000:1.0:1789760721.690677:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.690690:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.690700:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.690751:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.690756:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.690766:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.690785:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.690793:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.690884:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 76197us (79467us total) trans 4294972316 rc 0/0 00000100:00100000:1.0:1789760721.690894:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5028 00000100:00100000:1.0:1789760721.690902:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699967946440 time 700079962373 state 0x2 ref 0 00000100:00100000:3.0:1789760721.694345:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760721.694574:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515636992 00000100:00100000:0.0:1789760721.694584:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515636992 00000100:00100000:0.0:1789760721.694637:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 699967946440 time 700079962373 state 0x0 ref 1 00000100:00100000:0.0:1789760721.694652:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700080222789 time 700079962373 state 0x1 ref 1 00000100:00100000:0.0:1789760721.694677:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700080222789 time 700159966520 state 0x0 ref 1 00000100:00100000:0.0:1789760721.694688:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515636992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760721.694785:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20480000, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:0.0:1789760721.694794:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.694811:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760721.694816:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.694823:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.694837:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.734778:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20480000 -> 20484096 00010000:00010000:1.0:1789760721.734798:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.734812:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.734822:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.734829:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.734834:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.734845:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.734865:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.734878:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.734962:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515636992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40271us (40623us total) trans 4294972317 rc 0/0 00000100:00100000:1.0:1789760721.734972:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5029 00000100:00100000:1.0:1789760721.734982:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700080222789 time 700159966520 state 0x2 ref 0 00000001:00080000:2.0:1789760721.735342:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972317 is committed 00000100:00100000:0.0:1789760721.738145:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760721.738233:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637120 00000100:00100000:3.0:1789760721.738242:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637120 00000100:00100000:3.0:1789760721.738294:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700080222789 time 700159966520 state 0x0 ref 1 00000100:00100000:3.0:1789760721.738310:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700160226936 time 700159966520 state 0x1 ref 1 00000100:00100000:3.0:1789760721.738343:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700160226936 time 700203630773 state 0x0 ref 1 00000100:00100000:3.0:1789760721.738356:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515637120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760721.738404:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20484096, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:3.0:1789760721.738414:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.738430:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760721.738435:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.738442:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.738455:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.811491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972318 is committed 00002000:00010000:0.0:1789760721.811668:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20484096 -> 20488192 00010000:00010000:0.0:1789760721.811685:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.811699:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.811739:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.811747:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760721.811752:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.811761:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.811780:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.811787:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1db29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.811870:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515637120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73513us (73738us total) trans 4294972318 rc 0/0 00000100:00100000:0.0:1789760721.811880:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5030 00000100:00100000:0.0:1789760721.811890:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700160226936 time 700203630773 state 0x2 ref 0 00000100:00100000:3.0:1789760721.818268:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.818432:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637248 00000100:00100000:2.0:1789760721.818443:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637248 00000100:00100000:2.0:1789760721.818511:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700160226936 time 700203630773 state 0x0 ref 1 00000100:00100000:2.0:1789760721.818526:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700203891189 time 700203630773 state 0x1 ref 1 00000100:00100000:2.0:1789760721.818547:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700203891189 time 700283836551 state 0x0 ref 1 00000100:00100000:2.0:1789760721.818560:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515637248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760721.818612:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20488192, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xa007 00010000:00010000:2.0:1789760721.818621:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818638:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.818642:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.818648:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.818662:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.856219:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.856262:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.856315:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760721.856405:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637504 00000100:00100000:1.0:1789760721.856415:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637504 00000100:00100000:1.0:1789760721.856489:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637632 00000100:00100000:1.0:1789760721.856493:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637632 00000100:00100000:1.0:1789760721.856518:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 436 00000100:00100000:1.0:1789760721.856527:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515637504:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760721.856599:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515637504:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 73us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760721.856607:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 436 00000100:00100000:1.0:1789760721.856622:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 437 00000100:00100000:1.0:1789760721.856628:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806193800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515637632:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760721.856653:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806193800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515637632:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 25us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760721.856659:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 437 00000100:00100000:0.0:1789760721.857055:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637376 00000100:00100000:0.0:1789760721.857061:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637376 00000100:00100000:0.0:1789760721.857106:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 384 00000100:00100000:0.0:1789760721.857114:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909350450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515637376:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760721.857157:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909350450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515637376:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 44us (894us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760721.857163:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 384 00000100:00080000:2.0:1789760721.857477:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760721.893275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972319 is committed 00002000:00010000:1.0:1789760721.893361:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20488192 -> 20492288 00002000:00010000:1.0:1789760721.893365:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760721 -> 1789760722 00002000:00010000:1.0:1789760721.893367:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760721 -> 1789760722 00010000:00010000:1.0:1789760721.893378:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.893388:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.893395:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.893402:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760721.893407:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.893416:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.893434:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.893442:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.893564:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515637248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75001us (76628us total) trans 4294972319 rc 0/0 00000100:00100000:1.0:1789760721.893575:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5031 00000100:00100000:1.0:1789760721.893585:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700203891189 time 700283836551 state 0x2 ref 0 00000100:00100000:1.0:1789760721.898211:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.898292:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637760 00000100:00100000:2.0:1789760721.898303:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637760 00000100:00100000:2.0:1789760721.898414:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700203891189 time 700283836551 state 0x0 ref 1 00000100:00100000:2.0:1789760721.898430:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700284096967 time 700283836551 state 0x1 ref 1 00000100:00100000:2.0:1789760721.898460:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700284096967 time 700363749147 state 0x0 ref 1 00000100:00100000:2.0:1789760721.898472:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515637760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760721.898520:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20492288, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760721.898529:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.898551:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.898556:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.898562:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.898576:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760721.926670:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20492288 -> 20496384 00010000:00010000:3.0:1789760721.926692:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.926724:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.926744:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.926751:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760721.926756:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.926765:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.926793:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.926801:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.926896:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515637760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28421us (28690us total) trans 4294972320 rc 0/0 00000100:00100000:3.0:1789760721.926908:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5032 00000100:00100000:3.0:1789760721.926917:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700284096967 time 700363749147 state 0x2 ref 0 00000100:00100000:3.0:1789760721.930847:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760721.931148:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515637888 00000100:00100000:1.0:1789760721.931157:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515637888 00000100:00100000:1.0:1789760721.931208:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700284096967 time 700363749147 state 0x0 ref 1 00000100:00100000:1.0:1789760721.931223:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700364009563 time 700363749147 state 0x1 ref 1 00000100:00100000:1.0:1789760721.931249:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700364009563 time 700396538078 state 0x0 ref 1 00000100:00100000:1.0:1789760721.931260:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515637888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760721.931309:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20496384, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:1.0:1789760721.931330:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.931346:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.931351:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.931357:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.931372:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.938930:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972320 is committed 00000001:00080000:1.0:1789760721.984252:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972321 is committed 00002000:00010000:3.0:1789760721.984453:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20496384 -> 20500480 00010000:00010000:3.0:1789760721.984467:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.984480:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.984489:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.984567:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760721.984571:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.984579:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.984598:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.984605:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1db3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.984689:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515637888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53426us (53850us total) trans 4294972321 rc 0/0 00000100:00100000:3.0:1789760721.984698:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5033 00000100:00100000:3.0:1789760721.984740:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700364009563 time 700396538078 state 0x2 ref 0 00000100:00100000:1.0:1789760721.992587:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760721.992901:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638016 00000100:00100000:2.0:1789760721.992912:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638016 00000100:00100000:2.0:1789760721.992968:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700364009563 time 700396538078 state 0x0 ref 1 00000100:00100000:2.0:1789760721.992983:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700396798494 time 700396538078 state 0x1 ref 1 00000100:00100000:2.0:1789760721.993012:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700396798494 time 700458301568 state 0x0 ref 1 00000100:00100000:2.0:1789760721.993025:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515638016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760721.993081:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20500480, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760721.993090:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.993104:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.993109:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.993115:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.993145:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.035117:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972322 is committed 00002000:00010000:2.0:1789760722.035428:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20500480 -> 20504576 00010000:00010000:2.0:1789760722.035444:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.035458:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.035467:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.035475:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760722.035480:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.035488:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.035516:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.035523:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.035617:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515638016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42589us (43034us total) trans 4294972322 rc 0/0 00000100:00100000:2.0:1789760722.035628:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5034 00000100:00100000:2.0:1789760722.035639:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700396798494 time 700458301568 state 0x2 ref 0 00000100:00100000:1.0:1789760722.042911:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760722.043021:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638144 00000100:00100000:0.0:1789760722.043031:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638144 00000100:00100000:0.0:1789760722.043174:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700396798494 time 700458301568 state 0x0 ref 1 00000100:00100000:0.0:1789760722.043189:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700458561984 time 700458301568 state 0x1 ref 1 00000100:00100000:0.0:1789760722.043222:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700458561984 time 700508510928 state 0x0 ref 1 00000100:00100000:0.0:1789760722.043234:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515638144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760722.043285:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20504576, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:0.0:1789760722.043294:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.043312:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760722.043317:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.043323:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.043337:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760722.045106:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760722.105413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972323 is committed 00002000:00010000:2.0:1789760722.108433:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20504576 -> 20508672 00010000:00010000:2.0:1789760722.108453:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.108465:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.108474:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.108480:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760722.108484:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.108492:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.108511:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.108523:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1db4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.108629:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515638144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65392us (65723us total) trans 4294972323 rc 0/0 00000100:00100000:2.0:1789760722.108639:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5035 00000100:00100000:2.0:1789760722.108648:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700458561984 time 700508510928 state 0x2 ref 0 00000100:00100000:0.0:1789760722.121665:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.123596:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638272 00000100:00100000:3.0:1789760722.123610:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638272 00000100:00100000:0.0:1789760722.130686:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700458561984 time 700508510928 state 0x0 ref 1 00000100:00100000:0.0:1789760722.130720:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700508771344 time 700508510928 state 0x1 ref 1 00000100:00100000:0.0:1789760722.130768:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700508771344 time 700596056784 state 0x0 ref 1 00000100:00100000:0.0:1789760722.130783:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515638272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760722.130833:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20508672, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:0.0:1789760722.130842:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.130858:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760722.130863:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.130869:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.130883:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760722.214116:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20508672 -> 20512768 00010000:00010000:3.0:1789760722.214146:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.214160:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.214169:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.214177:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760722.214192:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.214202:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.214220:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.214228:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1db53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.214321:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515638272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 83538us (92660us total) trans 4294972324 rc 0/0 00000100:00100000:3.0:1789760722.214331:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5036 00000100:00100000:3.0:1789760722.214340:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700508771344 time 700596056784 state 0x2 ref 0 00000001:00080000:2.0:1789760722.214415:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972324 is committed 00000100:00100000:1.0:1789760722.218302:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.218499:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638400 00000100:00100000:3.0:1789760722.218510:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638400 00000100:00100000:3.0:1789760722.218562:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700508771344 time 700596056784 state 0x0 ref 1 00000100:00100000:3.0:1789760722.218577:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700596317200 time 700596056784 state 0x1 ref 1 00000100:00100000:3.0:1789760722.218612:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700596317200 time 700683901517 state 0x0 ref 1 00000100:00100000:3.0:1789760722.218624:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515638400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760722.218669:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20512768, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:3.0:1789760722.218677:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.218691:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760722.218696:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.218701:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.218745:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.239012:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760722.264024:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20512768 -> 20516864 00010000:00010000:0.0:1789760722.264043:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.264123:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.264143:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.264152:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760722.264158:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.264168:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.264187:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.264196:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1db5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.264288:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515638400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45661us (45991us total) trans 4294972325 rc 0/0 00000100:00100000:0.0:1789760722.264299:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5037 00000100:00100000:0.0:1789760722.264309:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700596317200 time 700683901517 state 0x2 ref 0 00000100:00100000:0.0:1789760722.267675:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.267903:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638528 00000100:00100000:3.0:1789760722.267914:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638528 00000100:00100000:3.0:1789760722.267970:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700596317200 time 700683901517 state 0x0 ref 1 00000100:00100000:3.0:1789760722.267985:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700684161933 time 700683901517 state 0x1 ref 1 00000100:00100000:3.0:1789760722.268016:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700684161933 time 700733304411 state 0x0 ref 1 00000100:00100000:3.0:1789760722.268029:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515638528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760722.268089:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20516864, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:3.0:1789760722.268098:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.268113:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760722.268118:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.268135:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.268150:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.271592:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972325 is committed 00000001:00080000:1.0:1789760722.296407:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972326 is committed 00002000:00010000:2.0:1789760722.296758:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20516864 -> 20520960 00010000:00010000:2.0:1789760722.296843:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.296856:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.296865:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.296872:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760722.296876:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.296884:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.296903:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.296911:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1db61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.297002:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515638528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28969us (29331us total) trans 4294972326 rc 0/0 00000100:00100000:2.0:1789760722.297011:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5038 00000100:00100000:2.0:1789760722.297020:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700684161933 time 700733304411 state 0x2 ref 0 00000100:00100000:3.0:1789760722.300937:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760722.301282:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638656 00000100:00100000:1.0:1789760722.301292:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638656 00000100:00100000:1.0:1789760722.301342:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700684161933 time 700733304411 state 0x0 ref 1 00000100:00100000:1.0:1789760722.301359:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700733564827 time 700733304411 state 0x1 ref 1 00000100:00100000:1.0:1789760722.301387:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700733564827 time 700766676694 state 0x0 ref 1 00000100:00100000:1.0:1789760722.301399:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515638656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760722.301441:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20520960, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:1.0:1789760722.301451:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.301466:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760722.301470:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.301476:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.301490:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.322001:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972327 is committed 00002000:00010000:3.0:1789760722.322187:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20520960 -> 20525056 00010000:00010000:3.0:1789760722.322203:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.322216:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.322225:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.322232:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760722.322237:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.322245:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.322265:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.322272:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.322359:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515638656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 20956us (21426us total) trans 4294972327 rc 0/0 00000100:00100000:3.0:1789760722.322370:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5039 00000100:00100000:3.0:1789760722.322380:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700733564827 time 700766676694 state 0x2 ref 0 00000100:00100000:1.0:1789760722.325002:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760722.325154:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638784 00000100:00100000:0.0:1789760722.325164:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638784 00000100:00100000:0.0:1789760722.325217:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700733564827 time 700766676694 state 0x0 ref 1 00000100:00100000:0.0:1789760722.325234:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700766937110 time 700766676694 state 0x1 ref 1 00000100:00100000:0.0:1789760722.325270:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700766937110 time 700790558824 state 0x0 ref 1 00000100:00100000:0.0:1789760722.325283:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515638784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760722.325334:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20525056, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:0.0:1789760722.325343:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.325490:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760722.325495:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.325502:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.325516:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.351542:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20525056 -> 20529152 00010000:00010000:2.0:1789760722.351561:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.351574:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.351582:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.351589:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760722.351594:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.351609:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.351629:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.351636:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1db6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.351760:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515638784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26474us (26763us total) trans 4294972328 rc 0/0 00000100:00100000:2.0:1789760722.351780:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5040 00000100:00100000:2.0:1789760722.351789:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700766937110 time 700790558824 state 0x2 ref 0 00000001:00080000:3.0:1789760722.354101:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972328 is committed 00000100:00100000:0.0:1789760722.357303:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760722.357448:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515638912 00000100:00100000:2.0:1789760722.357457:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515638912 00000100:00100000:2.0:1789760722.357508:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700766937110 time 700790558824 state 0x0 ref 1 00000100:00100000:2.0:1789760722.357523:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700790819240 time 700790558824 state 0x1 ref 1 00000100:00100000:2.0:1789760722.357556:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700790819240 time 700822845131 state 0x0 ref 1 00000100:00100000:2.0:1789760722.357568:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515638912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.357615:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20529152, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.357623:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.357639:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760722.357643:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.357661:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.357675:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.390230:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972329 is committed 00002000:00010000:2.0:1789760722.390780:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20529152 -> 20533248 00010000:00010000:2.0:1789760722.390797:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.390809:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.390818:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.390825:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760722.390830:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.390839:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.390859:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.390867:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.390997:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515638912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33425us (33697us total) trans 4294972329 rc 0/0 00000100:00100000:2.0:1789760722.391008:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5041 00000100:00100000:2.0:1789760722.391018:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700790819240 time 700822845131 state 0x2 ref 0 00000100:00100000:1.0:1789760722.395441:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.399528:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639040 00000100:00100000:3.0:1789760722.399544:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639040 00000100:00100000:2.0:1789760722.399673:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700790819240 time 700822845131 state 0x0 ref 1 00000100:00100000:2.0:1789760722.399689:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700823105547 time 700822845131 state 0x1 ref 1 00000100:00100000:2.0:1789760722.399756:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700823105547 time 700865044463 state 0x0 ref 1 00000100:00100000:2.0:1789760722.399770:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.399818:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20533248, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.399827:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.399842:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760722.399846:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.399852:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.399864:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.507583:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20533248 -> 20537344 00010000:00010000:1.0:1789760722.507603:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.507617:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.507626:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.507634:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760722.507639:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.507649:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.507670:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.507678:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1db7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.507893:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515639040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 108121us (112457us total) trans 4294972330 rc 0/0 00000100:00100000:1.0:1789760722.507905:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5042 00000100:00100000:1.0:1789760722.507915:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700823105547 time 700865044463 state 0x2 ref 0 00000001:00080000:1.0:1789760722.507957:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972330 is committed 00000100:00100000:1.0:1789760722.520854:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.521604:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639168 00000100:00100000:3.0:1789760722.521615:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639168 00000100:00100000:3.0:1789760722.521672:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700823105547 time 700865044463 state 0x0 ref 1 00000100:00100000:3.0:1789760722.521687:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700865304879 time 700865044463 state 0x1 ref 1 00000100:00100000:3.0:1789760722.521744:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700865304879 time 700987033299 state 0x0 ref 1 00000100:00100000:3.0:1789760722.521759:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760722.521818:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20537344, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:3.0:1789760722.521827:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.521842:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760722.521847:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.521853:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.521866:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.594507:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972331 is committed 00002000:00010000:1.0:1789760722.594876:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20537344 -> 20541440 00010000:00010000:1.0:1789760722.594891:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.594904:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.594914:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.594922:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760722.594927:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.595000:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.595026:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.595036:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1db84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.595250:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73489us (74400us total) trans 4294972331 rc 0/0 00000100:00100000:1.0:1789760722.595262:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5043 00000100:00100000:1.0:1789760722.595271:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700865304879 time 700987033299 state 0x2 ref 0 00000100:00100000:1.0:1789760722.608937:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.609020:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639296 00000100:00100000:3.0:1789760722.609030:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639296 00000100:00100000:3.0:1789760722.609090:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700865304879 time 700987033299 state 0x0 ref 1 00000100:00100000:3.0:1789760722.609104:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700987293715 time 700987033299 state 0x1 ref 1 00000100:00100000:3.0:1789760722.609141:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700987293715 time 701074430195 state 0x0 ref 1 00000100:00100000:3.0:1789760722.609154:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515639296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.609439:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20541440, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.609449:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.609465:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.609470:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.609476:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.609490:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.655285:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972332 is committed 00002000:00010000:0.0:1789760722.655365:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20541440 -> 20545536 00010000:00010000:0.0:1789760722.655377:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.655387:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.655393:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.655399:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760722.655402:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.655408:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.655422:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.655426:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1db8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.655509:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515639296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46352us (46575us total) trans 4294972332 rc 0/0 00000100:00100000:0.0:1789760722.655518:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5044 00000100:00100000:0.0:1789760722.655528:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700987293715 time 701074430195 state 0x2 ref 0 00000100:00100000:3.0:1789760722.658184:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760722.658267:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639424 00000100:00100000:1.0:1789760722.658276:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639424 00000100:00100000:1.0:1789760722.658333:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 700987293715 time 701074430195 state 0x0 ref 1 00000100:00100000:1.0:1789760722.658347:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701074690611 time 701074430195 state 0x1 ref 1 00000100:00100000:1.0:1789760722.658381:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701074690611 time 701123670360 state 0x0 ref 1 00000100:00100000:1.0:1789760722.658393:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760722.658436:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20545536, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:1.0:1789760722.658445:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.658460:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.658464:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.658470:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.658484:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760722.682328:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20545536 -> 20549632 00010000:00010000:3.0:1789760722.682360:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.682373:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.682387:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.682395:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760722.682400:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.682409:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.682429:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.682437:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1db92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.682533:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515639424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24136us (24352us total) trans 4294972333 rc 0/0 00000100:00100000:3.0:1789760722.682543:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5045 00000100:00100000:3.0:1789760722.682552:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701074690611 time 701123670360 state 0x2 ref 0 00000001:00080000:0.0:1789760722.683347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972333 is committed 00000100:00100000:1.0:1789760722.691631:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760722.691870:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639552 00000100:00100000:2.0:1789760722.691881:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639552 00000100:00100000:2.0:1789760722.691935:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701074690611 time 701123670360 state 0x0 ref 1 00000100:00100000:2.0:1789760722.691951:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701123930776 time 701123670360 state 0x1 ref 1 00000100:00100000:2.0:1789760722.692024:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701123930776 time 701157312624 state 0x0 ref 1 00000100:00100000:2.0:1789760722.692040:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.692103:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20549632, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.692114:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.692145:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.692150:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.692158:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.692172:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760722.737969:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20549632 -> 20553728 00010000:00010000:3.0:1789760722.737990:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.738014:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.738022:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.738029:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760722.738033:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.738042:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.738061:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.738068:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1db99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.738169:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515639552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46128us (46544us total) trans 4294972334 rc 0/0 00000100:00100000:3.0:1789760722.738180:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5046 00000100:00100000:3.0:1789760722.738189:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701123930776 time 701157312624 state 0x2 ref 0 00000001:00080000:0.0:1789760722.738601:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972334 is committed 00000100:00100000:1.0:1789760722.742063:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760722.742166:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639680 00000100:00100000:2.0:1789760722.742175:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639680 00000100:00100000:2.0:1789760722.742225:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701123930776 time 701157312624 state 0x0 ref 1 00000100:00100000:2.0:1789760722.742241:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701157573040 time 701157312624 state 0x1 ref 1 00000100:00100000:2.0:1789760722.742274:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701157573040 time 701207563439 state 0x0 ref 1 00000100:00100000:2.0:1789760722.742288:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515639680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.742337:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20553728, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.742346:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.742428:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.742433:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.742439:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.742459:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.770328:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972335 is committed 00002000:00010000:3.0:1789760722.770478:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20553728 -> 20557824 00010000:00010000:3.0:1789760722.770494:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.770508:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.770517:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.770525:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760722.770529:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.770538:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.770557:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.770565:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dba0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.770694:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515639680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28404us (28637us total) trans 4294972335 rc 0/0 00000100:00100000:3.0:1789760722.770704:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5047 00000100:00100000:3.0:1789760722.770750:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701157573040 time 701207563439 state 0x2 ref 0 00000100:00100000:1.0:1789760722.774673:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760722.774889:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639808 00000100:00100000:2.0:1789760722.774899:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639808 00000100:00100000:2.0:1789760722.774952:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701157573040 time 701207563439 state 0x0 ref 1 00000100:00100000:2.0:1789760722.774966:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701207823855 time 701207563439 state 0x1 ref 1 00000100:00100000:2.0:1789760722.775062:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701207823855 time 701240350109 state 0x0 ref 1 00000100:00100000:2.0:1789760722.775075:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.775136:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20557824, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.775145:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.775161:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.775165:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.775171:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.775185:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.803728:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972336 is committed 00002000:00010000:1.0:1789760722.803824:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20557824 -> 20561920 00010000:00010000:1.0:1789760722.803838:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.803852:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.803861:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.803868:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760722.803873:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.803883:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.803900:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.803907:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dba7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.804033:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28955us (29365us total) trans 4294972336 rc 0/0 00000100:00100000:1.0:1789760722.804050:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5048 00000100:00100000:1.0:1789760722.804059:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701207823855 time 701240350109 state 0x2 ref 0 00000100:00100000:1.0:1789760722.808521:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.808649:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515639936 00000100:00100000:3.0:1789760722.808659:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515639936 00000100:00100000:3.0:1789760722.808857:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701207823855 time 701240350109 state 0x0 ref 1 00000100:00100000:3.0:1789760722.808874:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701240610525 time 701240350109 state 0x1 ref 1 00000100:00100000:2.0:1789760722.808977:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701240610525 time 701274264008 state 0x0 ref 1 00000100:00100000:2.0:1789760722.808992:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.809048:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20561920, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.809057:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.809072:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.809077:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.809083:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.809097:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.835218:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972337 is committed 00002000:00010000:1.0:1789760722.835329:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20561920 -> 20566016 00010000:00010000:1.0:1789760722.835345:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.835370:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.835380:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.835388:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760722.835393:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.835402:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.835421:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.835429:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dbae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.835527:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515639936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26534us (27011us total) trans 4294972337 rc 0/0 00000100:00100000:1.0:1789760722.835538:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5049 00000100:00100000:1.0:1789760722.835547:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701240610525 time 701274264008 state 0x2 ref 0 00000100:00100000:0.0:1789760722.844645:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760722.845304:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640064 00000100:00100000:2.0:1789760722.845315:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640064 00000100:00100000:2.0:1789760722.845384:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701240610525 time 701274264008 state 0x0 ref 1 00000100:00100000:2.0:1789760722.845402:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701274524424 time 701274264008 state 0x1 ref 1 00000100:00100000:2.0:1789760722.845436:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701274524424 time 701310724519 state 0x0 ref 1 00000100:00100000:2.0:1789760722.845449:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.845505:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20566016, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xa007 00010000:00010000:2.0:1789760722.845515:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.845536:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.845540:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.845546:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.845560:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.883091:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20566016 -> 20570112 00002000:00010000:0.0:1789760722.883101:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760722 -> 1789760723 00002000:00010000:0.0:1789760722.883104:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760722 -> 1789760723 00010000:00010000:0.0:1789760722.883117:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.883141:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.883149:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.883157:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760722.883162:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.883171:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.883188:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.883196:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dbb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.883325:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515640064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37873us (38683us total) trans 4294972338 rc 0/0 00000100:00100000:0.0:1789760722.883335:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5050 00000100:00100000:0.0:1789760722.883345:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701274524424 time 701310724519 state 0x2 ref 0 00000001:00080000:1.0:1789760722.883531:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972338 is committed 00000100:00100000:3.0:1789760722.888663:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760722.889150:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640192 00000100:00100000:1.0:1789760722.889160:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640192 00000100:00100000:1.0:1789760722.889215:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701274524424 time 701310724519 state 0x0 ref 1 00000100:00100000:1.0:1789760722.889232:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701310984935 time 701310724519 state 0x1 ref 1 00000100:00100000:1.0:1789760722.889265:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701310984935 time 701354554302 state 0x0 ref 1 00000100:00100000:1.0:1789760722.889278:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515640192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760722.889328:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20570112, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760722.889337:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.889363:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.889368:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.889374:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.889388:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.913182:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20570112 -> 20574208 00000001:00080000:2.0:1789760722.913193:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972339 is committed 00010000:00010000:1.0:1789760722.913202:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.913214:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.913224:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.913231:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760722.913236:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.913249:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.913268:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.913276:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.913363:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515640192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24082us (24703us total) trans 4294972339 rc 0/0 00000100:00100000:1.0:1789760722.913379:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5051 00000100:00100000:1.0:1789760722.913389:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701310984935 time 701354554302 state 0x2 ref 0 00000100:00100000:3.0:1789760722.917994:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760722.918072:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640320 00000100:00100000:2.0:1789760722.918084:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640320 00000100:00100000:2.0:1789760722.918151:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701310984935 time 701354554302 state 0x0 ref 1 00000100:00100000:2.0:1789760722.918167:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701354814718 time 701354554302 state 0x1 ref 1 00000100:00100000:2.0:1789760722.918197:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701354814718 time 701383485714 state 0x0 ref 1 00000100:00100000:2.0:1789760722.918210:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760722.918259:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20574208, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:2.0:1789760722.918268:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.918286:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.918290:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.918297:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.918313:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.960331:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972340 is committed 00002000:00010000:1.0:1789760722.960595:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20574208 -> 20578304 00010000:00010000:1.0:1789760722.960610:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.960624:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.960633:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.960640:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760722.960646:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.960655:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.960675:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.960681:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1dbc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.960855:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42644us (42869us total) trans 4294972340 rc 0/0 00000100:00100000:1.0:1789760722.960864:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5052 00000100:00100000:1.0:1789760722.960871:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701354814718 time 701383485714 state 0x2 ref 0 00000100:00100000:0.0:1789760722.968462:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760722.968581:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640448 00000100:00100000:3.0:1789760722.968589:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640448 00000100:00100000:3.0:1789760722.968635:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701354814718 time 701383485714 state 0x0 ref 1 00000100:00100000:3.0:1789760722.968651:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701383746130 time 701383485714 state 0x1 ref 1 00000100:00100000:3.0:1789760722.968679:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701383746130 time 701433968070 state 0x0 ref 1 00000100:00100000:3.0:1789760722.968692:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760722.968791:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20578304, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:3.0:1789760722.968802:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.968819:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760722.968824:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.968830:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.968844:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.015322:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972341 is committed 00002000:00010000:1.0:1789760723.015518:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20578304 -> 20582400 00010000:00010000:1.0:1789760723.015535:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015548:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015557:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015564:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.015569:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015577:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015599:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015613:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dbca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.015749:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47016us (47253us total) trans 4294972341 rc 0/0 00000100:00100000:1.0:1789760723.015760:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5053 00000100:00100000:1.0:1789760723.015770:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701383746130 time 701433968070 state 0x2 ref 0 00000100:00100000:3.0:1789760723.026488:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.026598:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640576 00000100:00100000:1.0:1789760723.026609:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640576 00000100:00100000:1.0:1789760723.026662:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701383746130 time 701433968070 state 0x0 ref 1 00000100:00100000:1.0:1789760723.026678:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701434228486 time 701433968070 state 0x1 ref 1 00000100:00100000:1.0:1789760723.026743:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701434228486 time 701492031976 state 0x0 ref 1 00000100:00100000:1.0:1789760723.026757:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.026805:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20582400, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.026814:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.026829:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760723.026834:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.026840:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.026859:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.094812:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972342 is committed 00002000:00010000:1.0:1789760723.094930:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20582400 -> 20586496 00010000:00010000:1.0:1789760723.094945:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.094956:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.094964:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.094972:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.094976:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.094984:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.095002:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.095009:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.095243:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 68484us (68761us total) trans 4294972342 rc 0/0 00000100:00100000:1.0:1789760723.095253:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5054 00000100:00100000:1.0:1789760723.095262:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701434228486 time 701492031976 state 0x2 ref 0 00000100:00100000:0.0:1789760723.101674:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.101863:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640704 00000100:00100000:1.0:1789760723.101874:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640704 00000100:00100000:1.0:1789760723.101928:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701434228486 time 701492031976 state 0x0 ref 1 00000100:00100000:1.0:1789760723.101943:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701492292392 time 701492031976 state 0x1 ref 1 00000100:00100000:1.0:1789760723.101976:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701492292392 time 701567265585 state 0x0 ref 1 00000100:00100000:1.0:1789760723.101988:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.102036:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20586496, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.102106:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.102132:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760723.102136:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.102142:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.102223:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.142992:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20586496 -> 20590592 00010000:00010000:1.0:1789760723.143013:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.143025:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.143034:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.143108:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.143113:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.143130:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.143149:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.143229:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dbd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.143357:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972343 is committed 00000100:00100000:1.0:1789760723.143417:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41424us (41746us total) trans 4294972343 rc 0/0 00000100:00100000:1.0:1789760723.143432:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5055 00000100:00100000:1.0:1789760723.143442:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701492292392 time 701567265585 state 0x2 ref 0 00000100:00100000:2.0:1789760723.147108:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760723.147276:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640832 00000100:00100000:0.0:1789760723.147286:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640832 00000100:00100000:0.0:1789760723.147342:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701492292392 time 701567265585 state 0x0 ref 1 00000100:00100000:0.0:1789760723.147357:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701567526001 time 701567265585 state 0x1 ref 1 00000100:00100000:0.0:1789760723.147423:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701567526001 time 701612711600 state 0x0 ref 1 00000100:00100000:0.0:1789760723.147436:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760723.147483:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20590592, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:0.0:1789760723.147492:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.147508:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760723.147512:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.147519:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.147533:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.176801:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972344 is committed 00002000:00010000:1.0:1789760723.176901:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20590592 -> 20594688 00010000:00010000:1.0:1789760723.176917:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.176929:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.176938:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.176945:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.176950:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.176958:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.176977:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.176985:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dbdf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.177085:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515640832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29647us (29982us total) trans 4294972344 rc 0/0 00000100:00100000:1.0:1789760723.177095:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5056 00000100:00100000:1.0:1789760723.177104:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701567526001 time 701612711600 state 0x2 ref 0 00000100:00100000:2.0:1789760723.193814:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760723.196325:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515640960 00000100:00100000:3.0:1789760723.196385:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515640960 00000100:00100000:2.0:1789760723.196597:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701567526001 time 701612711600 state 0x0 ref 1 00000100:00100000:2.0:1789760723.196614:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701612972016 time 701612711600 state 0x1 ref 1 00000100:00100000:2.0:1789760723.196647:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701612972016 time 701661936807 state 0x0 ref 1 00000100:00100000:2.0:1789760723.196660:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515640960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760723.196724:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20594688, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:2.0:1789760723.196732:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.196748:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760723.196753:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.196759:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.196773:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.221656:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972345 is committed 00002000:00010000:1.0:1789760723.221797:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20594688 -> 20598784 00010000:00010000:1.0:1789760723.221814:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.221827:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.221837:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.221844:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.221849:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.221858:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.221877:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.221890:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dbe6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.221985:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515640960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25323us (28176us total) trans 4294972345 rc 0/0 00000100:00100000:1.0:1789760723.221996:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5057 00000100:00100000:1.0:1789760723.222007:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701612972016 time 701661936807 state 0x2 ref 0 00000100:00100000:0.0:1789760723.230785:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760723.230851:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641088 00000100:00100000:2.0:1789760723.230862:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641088 00000100:00100000:2.0:1789760723.230916:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701612972016 time 701661936807 state 0x0 ref 1 00000100:00100000:2.0:1789760723.230933:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701662197223 time 701661936807 state 0x1 ref 1 00000100:00100000:2.0:1789760723.230963:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701662197223 time 701696252841 state 0x0 ref 1 00000100:00100000:2.0:1789760723.230977:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515641088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760723.231026:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20598784, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:2.0:1789760723.231034:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.231049:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760723.231053:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.231059:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.231073:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.262029:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760723.272529:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20598784 -> 20602880 00010000:00010000:2.0:1789760723.272550:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.272570:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.272579:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.272587:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760723.272592:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.272601:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.272621:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.272630:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dbed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.272752:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515641088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41774us (41973us total) trans 4294972346 rc 0/0 00000100:00100000:2.0:1789760723.272773:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5058 00000100:00100000:2.0:1789760723.272784:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701662197223 time 701696252841 state 0x2 ref 0 00000001:00080000:0.0:1789760723.272922:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972346 is committed 00000100:00100000:2.0:1789760723.276282:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.276404:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641216 00000100:00100000:1.0:1789760723.276415:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641216 00000100:00100000:1.0:1789760723.276466:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701662197223 time 701696252841 state 0x0 ref 1 00000100:00100000:1.0:1789760723.276482:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701696513257 time 701696252841 state 0x1 ref 1 00000100:00100000:1.0:1789760723.276514:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701696513257 time 701741802720 state 0x0 ref 1 00000100:00100000:1.0:1789760723.276525:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515641216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.276572:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20602880, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.276581:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.276598:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760723.276602:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.276609:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.276624:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.307400:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20602880 -> 20606976 00010000:00010000:2.0:1789760723.307427:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.307441:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.307451:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.307459:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760723.307463:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.307471:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.307473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972347 is committed 00010000:00010000:2.0:1789760723.307488:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.307496:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dbf4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.307577:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515641216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31048us (31299us total) trans 4294972347 rc 0/0 00000100:00100000:2.0:1789760723.307588:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5059 00000100:00100000:2.0:1789760723.307597:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701696513257 time 701741802720 state 0x2 ref 0 00000100:00100000:1.0:1789760723.316024:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760723.316116:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641344 00000100:00100000:3.0:1789760723.316135:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641344 00000100:00100000:3.0:1789760723.316250:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701696513257 time 701741802720 state 0x0 ref 1 00000100:00100000:3.0:1789760723.316265:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701742063136 time 701741802720 state 0x1 ref 1 00000100:00100000:3.0:1789760723.316294:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701742063136 time 701781583350 state 0x0 ref 1 00000100:00100000:3.0:1789760723.316307:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515641344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760723.316354:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20606976, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:3.0:1789760723.316364:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.316380:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760723.316384:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.316390:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.316407:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.426860:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972348 is committed 00002000:00010000:3.0:1789760723.427026:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20606976 -> 20611072 00010000:00010000:3.0:1789760723.427054:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.427068:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.427077:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.427085:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760723.427091:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.427100:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.427130:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.427140:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dbfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.427244:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515641344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 110934us (111223us total) trans 4294972348 rc 0/0 00000100:00100000:3.0:1789760723.427255:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5060 00000100:00100000:3.0:1789760723.427265:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701742063136 time 701781583350 state 0x2 ref 0 00000100:00100000:0.0:1789760723.433818:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.433933:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641472 00000100:00100000:1.0:1789760723.433943:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641472 00000100:00100000:1.0:1789760723.433996:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701742063136 time 701781583350 state 0x0 ref 1 00000100:00100000:1.0:1789760723.434010:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701781843766 time 701781583350 state 0x1 ref 1 00000100:00100000:1.0:1789760723.434050:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701781843766 time 701899338647 state 0x0 ref 1 00000100:00100000:1.0:1789760723.434062:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515641472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.434108:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20611072, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.434128:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.434144:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760723.434148:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.434155:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.434177:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.454108:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760723.454144:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544408576:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760723.454213:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.454390:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408576 00000100:00100000:0.0:1789760723.454400:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408576 00000100:00100000:2.0:1789760723.454401:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760723.454413:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93b320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544408704:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760723.454451:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760723.454587:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408704 00000100:00100000:0.0:1789760723.454592:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408704 00000100:00100000:0.0:1789760723.454619:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1789760723.454628:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544408576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760723.454651:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3837788160 avail 3801088000 00000100:00100000:0.0:1789760723.454701:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544408576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (487us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.454839:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1789760723.454861:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1789760723.454869:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544408704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760723.454895:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760723.454943:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544408704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760723.454950:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00080000:0.0:1789760723.454967:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760723.454971:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760723.455015:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544408576:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760723.455041:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93b320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544408704:0@lo:13:osp-pre-1-0.0 00002000:00010000:0.0:1789760723.489482:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20611072 -> 20615168 00010000:00010000:0.0:1789760723.489503:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.489516:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.489531:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.489539:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760723.489544:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.489553:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.489571:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.489579:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1dc02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.489671:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515641472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55607us (55859us total) trans 4294972349 rc 0/0 00000100:00100000:0.0:1789760723.489681:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5061 00000100:00100000:0.0:1789760723.489689:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701781843766 time 701899338647 state 0x2 ref 0 00000001:00080000:0.0:1789760723.489757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972349 is committed 00000100:00100000:0.0:1789760723.493305:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.493393:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641600 00000100:00100000:1.0:1789760723.493403:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641600 00000100:00100000:1.0:1789760723.493452:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701781843766 time 701899338647 state 0x0 ref 1 00000100:00100000:1.0:1789760723.493468:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701899599063 time 701899338647 state 0x1 ref 1 00000100:00100000:1.0:1789760723.493498:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701899599063 time 701958788059 state 0x0 ref 1 00000100:00100000:1.0:1789760723.493510:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515641600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.493556:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20615168, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.493564:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.493579:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760723.493583:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.493589:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.493603:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.540823:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972350 is committed 00002000:00010000:1.0:1789760723.540908:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20615168 -> 20619264 00010000:00010000:1.0:1789760723.540924:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.540941:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.540949:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.540955:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.540959:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.540968:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.540989:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.540995:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.541135:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515641600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47622us (47834us total) trans 4294972350 rc 0/0 00000100:00100000:1.0:1789760723.541145:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5062 00000100:00100000:1.0:1789760723.541301:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701899599063 time 701958788059 state 0x2 ref 0 00000100:00100000:3.0:1789760723.545854:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.546572:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641728 00000100:00100000:1.0:1789760723.546583:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641728 00000100:00100000:1.0:1789760723.546636:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701899599063 time 701958788059 state 0x0 ref 1 00000100:00100000:1.0:1789760723.546651:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701959048475 time 701958788059 state 0x1 ref 1 00000100:00100000:1.0:1789760723.546685:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701959048475 time 702011974735 state 0x0 ref 1 00000100:00100000:1.0:1789760723.546697:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515641728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.546761:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20619264, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.546770:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.546785:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.546789:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.546795:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.546808:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.607802:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972351 is committed 00002000:00010000:1.0:1789760723.607905:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20619264 -> 20623360 00010000:00010000:1.0:1789760723.607923:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.607937:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.607952:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.607960:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.607965:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.607975:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.607994:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.608002:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.608625:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515641728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61924us (62774us total) trans 4294972351 rc 0/0 00000100:00100000:1.0:1789760723.608635:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5063 00000100:00100000:1.0:1789760723.608646:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701959048475 time 702011974735 state 0x2 ref 0 00000100:00100000:0.0:1789760723.617374:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760723.617747:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641856 00000100:00100000:2.0:1789760723.617866:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641856 00000100:00100000:2.0:1789760723.617929:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 701959048475 time 702011974735 state 0x0 ref 1 00000100:00100000:2.0:1789760723.617944:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702012235151 time 702011974735 state 0x1 ref 1 00000100:00100000:2.0:1789760723.617974:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702012235151 time 702083262396 state 0x0 ref 1 00000100:00100000:2.0:1789760723.617987:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515641856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760723.618102:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20623360, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:2.0:1789760723.618112:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.618137:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.618142:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.618149:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.618164:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.709909:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20623360 -> 20627456 00010000:00010000:1.0:1789760723.709930:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.709949:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.709958:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.709965:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.709970:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.709979:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.709999:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.710007:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dc17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.710125:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515641856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 92126us (92747us total) trans 4294972352 rc 0/0 00000100:00100000:1.0:1789760723.710137:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5064 00000100:00100000:1.0:1789760723.710148:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702012235151 time 702083262396 state 0x2 ref 0 00000001:00080000:0.0:1789760723.710559:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972352 is committed 00000100:00100000:0.0:1789760723.713962:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760723.714058:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515641984 00000100:00100000:2.0:1789760723.714068:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515641984 00000100:00100000:2.0:1789760723.714126:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702012235151 time 702083262396 state 0x0 ref 1 00000100:00100000:2.0:1789760723.714141:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702083522812 time 702083262396 state 0x1 ref 1 00000100:00100000:2.0:1789760723.714165:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702083522812 time 702179454873 state 0x0 ref 1 00000100:00100000:2.0:1789760723.714177:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515641984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760723.714220:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20627456, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:2.0:1789760723.714228:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.714243:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.714246:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.714252:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.714265:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.759425:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972353 is committed 00002000:00010000:0.0:1789760723.759585:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20627456 -> 20631552 00010000:00010000:0.0:1789760723.759601:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.759615:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.759624:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.759631:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760723.759636:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.759645:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.759665:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.759674:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.759799:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515641984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45618us (45840us total) trans 4294972353 rc 0/0 00000100:00100000:0.0:1789760723.759809:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5065 00000100:00100000:0.0:1789760723.759819:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702083522812 time 702179454873 state 0x2 ref 0 00000100:00100000:3.0:1789760723.763615:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760723.763739:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642112 00000100:00100000:0.0:1789760723.763750:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642112 00000100:00100000:0.0:1789760723.763812:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702083522812 time 702179454873 state 0x0 ref 1 00000100:00100000:0.0:1789760723.763829:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702179715289 time 702179454873 state 0x1 ref 1 00000100:00100000:0.0:1789760723.763850:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702179715289 time 702229139332 state 0x0 ref 1 00000100:00100000:0.0:1789760723.763863:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515642112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760723.763908:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20631552, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:0.0:1789760723.763916:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.763933:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760723.763937:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.763944:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.763957:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.810323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972354 is committed 00002000:00010000:0.0:1789760723.810348:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20631552 -> 20635648 00010000:00010000:0.0:1789760723.810365:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.810379:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.810388:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.810395:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760723.810400:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.810408:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.810427:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.810434:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dc25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.810570:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515642112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46703us (46959us total) trans 4294972354 rc 0/0 00000100:00100000:0.0:1789760723.810581:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5066 00000100:00100000:0.0:1789760723.810589:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702179715289 time 702229139332 state 0x2 ref 0 00000100:00100000:0.0:1789760723.813214:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.813301:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642240 00000100:00100000:1.0:1789760723.813311:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642240 00000100:00100000:1.0:1789760723.813377:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702179715289 time 702229139332 state 0x0 ref 1 00000100:00100000:1.0:1789760723.813391:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702229399748 time 702229139332 state 0x1 ref 1 00000100:00100000:1.0:1789760723.813418:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702229399748 time 702278708092 state 0x0 ref 1 00000100:00100000:1.0:1789760723.813429:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515642240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.813473:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20635648, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xa007 00010000:00010000:1.0:1789760723.813481:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.813503:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.813507:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.813512:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.813525:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.845017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972355 is committed 00002000:00010000:1.0:1789760723.845264:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20635648 -> 20639744 00002000:00010000:1.0:1789760723.845270:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760723 -> 1789760724 00002000:00010000:1.0:1789760723.845273:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760723 -> 1789760724 00010000:00010000:1.0:1789760723.845287:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.845307:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.845316:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.845323:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.845327:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.845336:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.845355:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.845363:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.845451:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515642240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32019us (32240us total) trans 4294972355 rc 0/0 00000100:00100000:1.0:1789760723.845462:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5067 00000100:00100000:1.0:1789760723.845471:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702229399748 time 702278708092 state 0x2 ref 0 00000100:00100000:2.0:1789760723.850824:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.851044:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642368 00000100:00100000:1.0:1789760723.851055:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642368 00000100:00100000:1.0:1789760723.851110:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702229399748 time 702278708092 state 0x0 ref 1 00000100:00100000:1.0:1789760723.851266:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702278968508 time 702278708092 state 0x1 ref 1 00000100:00100000:1.0:1789760723.851303:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702278968508 time 702316591142 state 0x0 ref 1 00000100:00100000:1.0:1789760723.851317:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515642368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.851376:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20639744, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:1.0:1789760723.851386:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.851402:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.851407:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.851413:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.851427:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.895917:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20639744 -> 20643840 00010000:00010000:1.0:1789760723.895948:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.895962:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.895977:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.895984:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.895989:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.895997:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.896016:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.896023:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.896133:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515642368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44815us (45312us total) trans 4294972356 rc 0/0 00000100:00100000:1.0:1789760723.896143:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5068 00000100:00100000:1.0:1789760723.896151:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702278968508 time 702316591142 state 0x2 ref 0 00000001:00080000:2.0:1789760723.897013:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972356 is committed 00000100:00100000:3.0:1789760723.901149:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760723.901323:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642496 00000100:00100000:1.0:1789760723.901333:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642496 00000100:00100000:1.0:1789760723.901385:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702278968508 time 702316591142 state 0x0 ref 1 00000100:00100000:1.0:1789760723.901400:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702316851558 time 702316591142 state 0x1 ref 1 00000100:00100000:1.0:1789760723.901552:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702316851558 time 702366839719 state 0x0 ref 1 00000100:00100000:1.0:1789760723.901565:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515642496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760723.901609:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20643840, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:1.0:1789760723.901618:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.901632:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.901637:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.901643:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.901657:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.932968:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20643840 -> 20647936 00010000:00010000:1.0:1789760723.932988:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.933002:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.933011:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.933030:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760723.933042:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.933052:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.933072:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.933080:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.933182:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515642496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31614us (32036us total) trans 4294972357 rc 0/0 00000100:00100000:1.0:1789760723.933193:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5069 00000100:00100000:1.0:1789760723.933201:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702316851558 time 702366839719 state 0x2 ref 0 00000001:00080000:2.0:1789760723.933270:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972357 is committed 00000100:00100000:3.0:1789760723.938864:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760723.938956:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642624 00000100:00100000:2.0:1789760723.938965:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642624 00000100:00100000:2.0:1789760723.939014:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702316851558 time 702366839719 state 0x0 ref 1 00000100:00100000:2.0:1789760723.939028:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702367100135 time 702366839719 state 0x1 ref 1 00000100:00100000:2.0:1789760723.939053:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702367100135 time 702404343290 state 0x0 ref 1 00000100:00100000:2.0:1789760723.939064:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515642624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760723.939132:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20647936, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:2.0:1789760723.939141:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.939155:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.939159:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.939164:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.939177:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.969348:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972358 is committed 00002000:00010000:2.0:1789760723.969512:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20647936 -> 20652032 00010000:00010000:2.0:1789760723.969527:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.969540:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.969549:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.969557:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760723.969562:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.969571:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.969592:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.969599:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.969952:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515642624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30885us (31091us total) trans 4294972358 rc 0/0 00000100:00100000:2.0:1789760723.969964:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5070 00000100:00100000:2.0:1789760723.969973:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702367100135 time 702404343290 state 0x2 ref 0 00000100:00100000:0.0:1789760723.976271:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760723.976624:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642752 00000100:00100000:2.0:1789760723.976634:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642752 00000100:00100000:2.0:1789760723.976690:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702367100135 time 702404343290 state 0x0 ref 1 00000100:00100000:2.0:1789760723.976862:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702404603706 time 702404343290 state 0x1 ref 1 00000100:00100000:2.0:1789760723.976994:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702404603706 time 702442281237 state 0x0 ref 1 00000100:00100000:2.0:1789760723.977008:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515642752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760723.977067:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20652032, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:2.0:1789760723.977077:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.977091:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.977096:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.977103:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.977126:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.006674:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972359 is committed 00002000:00010000:3.0:1789760724.006879:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20652032 -> 20656128 00010000:00010000:3.0:1789760724.006895:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.006907:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.006916:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.006931:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760724.006936:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.006944:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.006964:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.006972:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1dc48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.007071:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515642752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30061us (30805us total) trans 4294972359 rc 0/0 00000100:00100000:3.0:1789760724.007082:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5071 00000100:00100000:3.0:1789760724.007092:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702404603706 time 702442281237 state 0x2 ref 0 00000100:00100000:0.0:1789760724.012703:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760724.012975:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515642880 00000100:00100000:3.0:1789760724.012985:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515642880 00000100:00100000:3.0:1789760724.013066:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702404603706 time 702442281237 state 0x0 ref 1 00000100:00100000:3.0:1789760724.013083:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702442541653 time 702442281237 state 0x1 ref 1 00000100:00100000:3.0:1789760724.013104:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702442541653 time 702478393360 state 0x0 ref 1 00000100:00100000:3.0:1789760724.013130:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515642880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.013177:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20656128, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:3.0:1789760724.013186:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.013202:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760724.013206:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.013212:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.013227:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760724.013656:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760724.047160:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20656128 -> 20660224 00010000:00010000:3.0:1789760724.047182:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047195:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047205:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047212:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760724.047216:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047226:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047245:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.047253:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.047340:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515642880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34209us (34643us total) trans 4294972360 rc 0/0 00000100:00100000:3.0:1789760724.047352:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5072 00000100:00100000:3.0:1789760724.047361:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702442541653 time 702478393360 state 0x2 ref 0 00000001:00080000:1.0:1789760724.047932:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972360 is committed 00000100:00100000:0.0:1789760724.050941:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760724.051017:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643008 00000100:00100000:1.0:1789760724.051027:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643008 00000100:00100000:1.0:1789760724.051149:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702442541653 time 702478393360 state 0x0 ref 1 00000100:00100000:1.0:1789760724.051164:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702478653776 time 702478393360 state 0x1 ref 1 00000100:00100000:1.0:1789760724.051194:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702478653776 time 702516482688 state 0x0 ref 1 00000100:00100000:1.0:1789760724.051207:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515643008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760724.051254:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20660224, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:1.0:1789760724.051263:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.051278:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760724.051283:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.051289:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.051304:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.085885:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972361 is committed 00002000:00010000:0.0:1789760724.085997:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20660224 -> 20664320 00010000:00010000:0.0:1789760724.086013:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.086025:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.086048:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.086055:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760724.086060:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.086069:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.086088:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.086096:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dc56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.086202:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515643008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34993us (35267us total) trans 4294972361 rc 0/0 00000100:00100000:0.0:1789760724.086212:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5073 00000100:00100000:0.0:1789760724.086222:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702478653776 time 702516482688 state 0x2 ref 0 00000100:00100000:0.0:1789760724.089981:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760724.090065:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643136 00000100:00100000:1.0:1789760724.090074:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643136 00000100:00100000:1.0:1789760724.090134:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702478653776 time 702516482688 state 0x0 ref 1 00000100:00100000:1.0:1789760724.090147:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702516743104 time 702516482688 state 0x1 ref 1 00000100:00100000:1.0:1789760724.090209:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702516743104 time 702555497759 state 0x0 ref 1 00000100:00100000:1.0:1789760724.090222:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515643136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760724.090268:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20664320, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:1.0:1789760724.090277:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.090293:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760724.090298:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.090304:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.090318:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.126330:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20664320 -> 20668416 00010000:00010000:0.0:1789760724.126352:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.126376:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.126385:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.126393:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760724.126397:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.126401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972362 is committed 00010000:00010000:0.0:1789760724.126513:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.126533:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.126547:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dc5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.126641:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515643136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36416us (36664us total) trans 4294972362 rc 0/0 00000100:00100000:0.0:1789760724.126652:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5074 00000100:00100000:0.0:1789760724.126663:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702516743104 time 702555497759 state 0x2 ref 0 00000100:00100000:3.0:1789760724.130298:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760724.130456:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643264 00000100:00100000:1.0:1789760724.130465:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643264 00000100:00100000:1.0:1789760724.130520:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702516743104 time 702555497759 state 0x0 ref 1 00000100:00100000:1.0:1789760724.130536:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702555758175 time 702555497759 state 0x1 ref 1 00000100:00100000:1.0:1789760724.130569:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702555758175 time 702595857817 state 0x0 ref 1 00000100:00100000:1.0:1789760724.130582:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515643264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760724.130626:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20668416, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:1.0:1789760724.130634:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.130648:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760724.130652:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.130658:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.130670:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.168842:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20668416 -> 20672512 00010000:00010000:1.0:1789760724.168861:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.168873:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.168881:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.168887:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760724.168892:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.168902:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.168921:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.168928:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dc64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.169016:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515643264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38432us (38724us total) trans 4294972363 rc 0/0 00000100:00100000:1.0:1789760724.169040:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5075 00000100:00100000:1.0:1789760724.169050:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702555758175 time 702595857817 state 0x2 ref 0 00000001:00080000:3.0:1789760724.169392:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972363 is committed 00000100:00100000:3.0:1789760724.178380:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760724.181658:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643392 00000100:00100000:2.0:1789760724.181679:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643392 00000100:00100000:3.0:1789760724.184320:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702555758175 time 702595857817 state 0x0 ref 1 00000100:00100000:3.0:1789760724.184339:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702596118233 time 702595857817 state 0x1 ref 1 00000100:00100000:3.0:1789760724.184369:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702596118233 time 702649658667 state 0x0 ref 1 00000100:00100000:3.0:1789760724.184383:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515643392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.184432:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20672512, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:3.0:1789760724.184441:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.184455:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760724.184460:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.184466:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.184479:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.287615:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760724.302091:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20672512 -> 20676608 00010000:00010000:2.0:1789760724.302131:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.302143:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.302151:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.302159:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760724.302164:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.302172:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.302191:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.302199:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.302293:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515643392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 117909us (123916us total) trans 4294972364 rc 0/0 00000100:00100000:2.0:1789760724.302303:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5076 00000100:00100000:2.0:1789760724.302312:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702596118233 time 702649658667 state 0x2 ref 0 00000001:00080000:3.0:1789760724.302904:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972364 is committed 00000100:00100000:0.0:1789760724.309760:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760724.309885:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643520 00000100:00100000:3.0:1789760724.309895:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643520 00000100:00100000:3.0:1789760724.309953:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702596118233 time 702649658667 state 0x0 ref 1 00000100:00100000:3.0:1789760724.309968:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702649919083 time 702649658667 state 0x1 ref 1 00000100:00100000:3.0:1789760724.310029:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702649919083 time 702775317874 state 0x0 ref 1 00000100:00100000:3.0:1789760724.310042:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515643520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.310087:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20676608, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:3.0:1789760724.310096:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.311339:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760724.311344:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.311352:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.311368:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.406164:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972365 is committed 00002000:00010000:2.0:1789760724.406323:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20676608 -> 20680704 00010000:00010000:2.0:1789760724.406338:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406349:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406358:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406366:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760724.406370:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406385:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406405:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.406412:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1dc72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.406502:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515643520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 96457us (96747us total) trans 4294972365 rc 0/0 00000100:00100000:2.0:1789760724.406512:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5077 00000100:00100000:2.0:1789760724.406522:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702649919083 time 702775317874 state 0x2 ref 0 00000100:00100000:0.0:1789760724.416469:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760724.416558:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643648 00000100:00100000:3.0:1789760724.416569:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643648 00000100:00100000:3.0:1789760724.416622:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702649919083 time 702775317874 state 0x0 ref 1 00000100:00100000:3.0:1789760724.416638:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702775578290 time 702775317874 state 0x1 ref 1 00000100:00100000:3.0:1789760724.416673:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702775578290 time 702881961464 state 0x0 ref 1 00000100:00100000:3.0:1789760724.416686:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515643648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.416778:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20680704, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:3.0:1789760724.416788:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.416803:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760724.416807:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.416813:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.416828:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.482757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972366 is committed 00002000:00010000:2.0:1789760724.483108:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20680704 -> 20684800 00010000:00010000:2.0:1789760724.483132:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.483146:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.483155:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.483162:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760724.483167:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.483177:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.483197:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.483204:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1dc79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.483304:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515643648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66615us (66840us total) trans 4294972366 rc 0/0 00000100:00100000:2.0:1789760724.483314:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5078 00000100:00100000:2.0:1789760724.483324:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702775578290 time 702881961464 state 0x2 ref 0 00000100:00100000:3.0:1789760724.493417:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760724.493959:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643776 00000100:00100000:0.0:1789760724.493968:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643776 00000100:00100000:0.0:1789760724.494026:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702775578290 time 702881961464 state 0x0 ref 1 00000100:00100000:0.0:1789760724.494045:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702882221880 time 702881961464 state 0x1 ref 1 00000100:00100000:0.0:1789760724.494080:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702882221880 time 702959369486 state 0x0 ref 1 00000100:00100000:0.0:1789760724.494092:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515643776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760724.494154:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20684800, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:0.0:1789760724.494163:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.494176:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760724.494181:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.494187:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.494200:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.584158:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972367 is committed 00002000:00010000:3.0:1789760724.584326:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20684800 -> 20688896 00010000:00010000:3.0:1789760724.584343:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.584355:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.584365:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.584372:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760724.584377:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.584386:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.584407:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.584415:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dc80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.584509:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515643776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 90414us (91097us total) trans 4294972367 rc 0/0 00000100:00100000:3.0:1789760724.584519:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5079 00000100:00100000:3.0:1789760724.584529:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702882221880 time 702959369486 state 0x2 ref 0 00000100:00100000:2.0:1789760724.588377:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760724.588469:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515643904 00000100:00100000:1.0:1789760724.588478:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515643904 00000100:00100000:1.0:1789760724.588529:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702882221880 time 702959369486 state 0x0 ref 1 00000100:00100000:1.0:1789760724.588544:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702959629902 time 702959369486 state 0x1 ref 1 00000100:00100000:1.0:1789760724.588571:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702959629902 time 703053860459 state 0x0 ref 1 00000100:00100000:1.0:1789760724.588584:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515643904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760724.588632:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20688896, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:1.0:1789760724.588641:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.588656:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760724.588660:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.588666:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.588679:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.658968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972368 is committed 00002000:00010000:3.0:1789760724.659141:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20688896 -> 20692992 00010000:00010000:3.0:1789760724.659157:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.659169:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.659178:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.659185:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760724.659189:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.659197:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.659215:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.659227:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1dc87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.659318:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515643904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70732us (70945us total) trans 4294972368 rc 0/0 00000100:00100000:3.0:1789760724.659329:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5080 00000100:00100000:3.0:1789760724.659338:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702959629902 time 703053860459 state 0x2 ref 0 00000100:00100000:1.0:1789760724.668221:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760724.668317:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644032 00000100:00100000:3.0:1789760724.668327:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644032 00000100:00100000:3.0:1789760724.668382:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 702959629902 time 703053860459 state 0x0 ref 1 00000100:00100000:3.0:1789760724.668398:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703054120875 time 703053860459 state 0x1 ref 1 00000100:00100000:3.0:1789760724.668432:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703054120875 time 703133720763 state 0x0 ref 1 00000100:00100000:3.0:1789760724.668444:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515644032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.668490:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20692992, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:3.0:1789760724.668499:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668515:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760724.668519:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.668526:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.668544:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.725461:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972369 is committed 00002000:00010000:3.0:1789760724.725631:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20692992 -> 20697088 00010000:00010000:3.0:1789760724.725646:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.725659:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.725669:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.725675:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760724.725681:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.725690:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.725725:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.725733:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dc8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.725825:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515644032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57379us (57610us total) trans 4294972369 rc 0/0 00000100:00100000:3.0:1789760724.725836:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5081 00000100:00100000:3.0:1789760724.725845:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703054120875 time 703133720763 state 0x2 ref 0 00000100:00100000:0.0:1789760724.728478:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760724.728562:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644160 00000100:00100000:2.0:1789760724.728572:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644160 00000100:00100000:2.0:1789760724.728625:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703054120875 time 703133720763 state 0x0 ref 1 00000100:00100000:2.0:1789760724.728640:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703133981179 time 703133720763 state 0x1 ref 1 00000100:00100000:2.0:1789760724.728669:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703133981179 time 703193957775 state 0x0 ref 1 00000100:00100000:2.0:1789760724.728681:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515644160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760724.728768:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20697088, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:2.0:1789760724.728777:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.728792:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.728796:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.728802:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.728815:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.800079:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20697088 -> 20701184 00010000:00010000:2.0:1789760724.800100:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.800122:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.800131:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.800138:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760724.800144:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.800153:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.800174:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.800182:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.800278:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515644160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71594us (71803us total) trans 4294972370 rc 0/0 00000100:00100000:2.0:1789760724.800288:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5082 00000100:00100000:2.0:1789760724.800298:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703133981179 time 703193957775 state 0x2 ref 0 00000001:00080000:1.0:1789760724.800990:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972370 is committed 00000100:00100000:1.0:1789760724.804995:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760724.805074:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644288 00000100:00100000:2.0:1789760724.805088:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644288 00000100:00100000:2.0:1789760724.805151:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703133981179 time 703193957775 state 0x0 ref 1 00000100:00100000:2.0:1789760724.805166:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703194218191 time 703193957775 state 0x1 ref 1 00000100:00100000:2.0:1789760724.805194:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703194218191 time 703270483640 state 0x0 ref 1 00000100:00100000:2.0:1789760724.805206:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515644288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760724.805250:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20701184, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:2.0:1789760724.805259:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.805274:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.805278:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.805284:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.805299:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.849330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972371 is committed 00002000:00010000:2.0:1789760724.849411:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20701184 -> 20705280 00010000:00010000:2.0:1789760724.849427:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.849440:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.849450:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.849458:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760724.849462:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.849471:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.849490:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.849503:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dc9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.849630:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515644288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44420us (44638us total) trans 4294972371 rc 0/0 00000100:00100000:2.0:1789760724.849641:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5083 00000100:00100000:2.0:1789760724.849652:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703194218191 time 703270483640 state 0x2 ref 0 00000100:00100000:0.0:1789760724.852301:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760724.852417:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644416 00000100:00100000:2.0:1789760724.852426:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644416 00000100:00100000:2.0:1789760724.852478:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703194218191 time 703270483640 state 0x0 ref 1 00000100:00100000:2.0:1789760724.852492:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703270744056 time 703270483640 state 0x1 ref 1 00000100:00100000:2.0:1789760724.852558:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703270744056 time 703317847324 state 0x0 ref 1 00000100:00100000:2.0:1789760724.852571:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515644416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760724.852614:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20705280, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xa007 00010000:00010000:2.0:1789760724.852622:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.852638:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.852642:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.852647:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.852667:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.895093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972372 is committed 00002000:00010000:0.0:1789760724.895185:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20705280 -> 20709376 00002000:00010000:0.0:1789760724.895190:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760724 -> 1789760725 00002000:00010000:0.0:1789760724.895193:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760724 -> 1789760725 00010000:00010000:0.0:1789760724.895206:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.895219:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.895228:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.895236:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760724.895241:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.895250:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.895274:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.895282:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dca3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.895410:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515644416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42837us (43114us total) trans 4294972372 rc 0/0 00000100:00100000:0.0:1789760724.895422:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5084 00000100:00100000:0.0:1789760724.895432:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703270744056 time 703317847324 state 0x2 ref 0 00000100:00100000:1.0:1789760724.899659:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760724.899741:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644544 00000100:00100000:3.0:1789760724.899756:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644544 00000100:00100000:3.0:1789760724.899809:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703270744056 time 703317847324 state 0x0 ref 1 00000100:00100000:3.0:1789760724.899823:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703318107740 time 703317847324 state 0x1 ref 1 00000100:00100000:3.0:1789760724.899891:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703318107740 time 703365180404 state 0x0 ref 1 00000100:00100000:3.0:1789760724.899903:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515644544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.899949:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20709376, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:3.0:1789760724.899958:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.899972:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760724.899977:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.899983:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.900004:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760724.942899:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20709376 -> 20713472 00010000:00010000:3.0:1789760724.942918:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.942932:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.942941:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.942948:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760724.942953:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.942961:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.942980:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.942988:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.943097:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515644544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43189us (43439us total) trans 4294972373 rc 0/0 00000100:00100000:3.0:1789760724.943118:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5085 00000100:00100000:3.0:1789760724.943128:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703318107740 time 703365180404 state 0x2 ref 0 00000001:00080000:0.0:1789760724.944205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972373 is committed 00000100:00100000:1.0:1789760724.951262:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760724.951309:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644672 00000100:00100000:3.0:1789760724.951319:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644672 00000100:00100000:3.0:1789760724.951373:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703318107740 time 703365180404 state 0x0 ref 1 00000100:00100000:3.0:1789760724.951388:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703365440820 time 703365180404 state 0x1 ref 1 00000100:00100000:3.0:1789760724.951415:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703365440820 time 703416704756 state 0x0 ref 1 00000100:00100000:3.0:1789760724.951424:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515644672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760724.951466:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20713472, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:3.0:1789760724.951475:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.951491:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760724.951496:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.951502:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.951516:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.995351:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972374 is committed 00002000:00010000:1.0:1789760724.995641:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20713472 -> 20717568 00010000:00010000:1.0:1789760724.995659:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.995671:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.995681:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.995688:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760724.995693:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.995701:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.995751:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.995764:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1dcb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.995844:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515644672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44415us (44584us total) trans 4294972374 rc 0/0 00000100:00100000:1.0:1789760724.995853:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5086 00000100:00100000:1.0:1789760724.995862:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703365440820 time 703416704756 state 0x2 ref 0 00000100:00100000:1.0:1789760725.002635:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760725.002790:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644800 00000100:00100000:3.0:1789760725.002800:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644800 00000100:00100000:3.0:1789760725.002849:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703365440820 time 703416704756 state 0x0 ref 1 00000100:00100000:3.0:1789760725.002865:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703416965172 time 703416704756 state 0x1 ref 1 00000100:00100000:3.0:1789760725.002932:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703416965172 time 703468220394 state 0x0 ref 1 00000100:00100000:3.0:1789760725.002945:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515644800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760725.002992:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20717568, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:3.0:1789760725.003001:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.003017:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760725.003031:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.003038:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.003051:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.042477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972375 is committed 00002000:00010000:3.0:1789760725.042548:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20717568 -> 20721664 00010000:00010000:3.0:1789760725.042564:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.042582:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.042659:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.042667:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760725.042672:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.042681:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.042701:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.042735:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dcb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.042819:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515644800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39872us (40187us total) trans 4294972375 rc 0/0 00000100:00100000:3.0:1789760725.042829:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5087 00000100:00100000:3.0:1789760725.042839:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703416965172 time 703468220394 state 0x2 ref 0 00000100:00100000:0.0:1789760725.053901:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760725.054048:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515644928 00000100:00100000:2.0:1789760725.054231:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515644928 00000100:00100000:2.0:1789760725.054282:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703416965172 time 703468220394 state 0x0 ref 1 00000100:00100000:2.0:1789760725.054298:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703468480810 time 703468220394 state 0x1 ref 1 00000100:00100000:2.0:1789760725.054367:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703468480810 time 703519654903 state 0x0 ref 1 00000100:00100000:2.0:1789760725.054380:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515644928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760725.054426:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20721664, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:2.0:1789760725.054435:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.054450:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760725.054455:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.054461:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.054479:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.102408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972376 is committed 00002000:00010000:0.0:1789760725.102612:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20721664 -> 20725760 00010000:00010000:0.0:1789760725.102628:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.102641:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.102651:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.102658:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760725.102664:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.102673:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.102695:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.102703:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1dcbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.102819:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515644928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48435us (48920us total) trans 4294972376 rc 0/0 00000100:00100000:0.0:1789760725.102830:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5088 00000100:00100000:0.0:1789760725.102839:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703468480810 time 703519654903 state 0x2 ref 0 00000100:00100000:3.0:1789760725.106570:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760725.106828:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645056 00000100:00100000:2.0:1789760725.106839:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645056 00000100:00100000:2.0:1789760725.106893:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703468480810 time 703519654903 state 0x0 ref 1 00000100:00100000:2.0:1789760725.106909:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703519915319 time 703519654903 state 0x1 ref 1 00000100:00100000:2.0:1789760725.106986:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703519915319 time 703572274750 state 0x0 ref 1 00000100:00100000:2.0:1789760725.106998:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515645056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760725.107045:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20725760, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:2.0:1789760725.107054:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.107069:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760725.107074:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.107080:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.107094:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.167485:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972377 is committed 00002000:00010000:0.0:1789760725.167599:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20725760 -> 20729856 00010000:00010000:0.0:1789760725.167613:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.167625:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.167635:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.167647:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760725.167652:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.167661:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.167679:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.167685:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1dcc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.167873:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515645056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60872us (61307us total) trans 4294972377 rc 0/0 00000100:00100000:0.0:1789760725.167884:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5089 00000100:00100000:0.0:1789760725.167893:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703519915319 time 703572274750 state 0x2 ref 0 00000100:00100000:1.0:1789760725.171306:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760725.171599:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645184 00000100:00100000:0.0:1789760725.171609:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645184 00000100:00100000:0.0:1789760725.171660:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703519915319 time 703572274750 state 0x0 ref 1 00000100:00100000:0.0:1789760725.171674:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703572535166 time 703572274750 state 0x1 ref 1 00000100:00100000:0.0:1789760725.172014:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703572535166 time 703637301137 state 0x0 ref 1 00000100:00100000:0.0:1789760725.172030:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515645184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760725.172076:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20729856, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:0.0:1789760725.172085:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.172100:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760725.172118:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.172125:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.172139:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.215865:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20729856 -> 20733952 00010000:00010000:1.0:1789760725.215884:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.215896:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.215904:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.215912:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760725.215916:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.215926:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.215945:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.215953:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dccd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.216046:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515645184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44013us (44744us total) trans 4294972378 rc 0/0 00000100:00100000:1.0:1789760725.216056:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5090 00000100:00100000:1.0:1789760725.216066:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703572535166 time 703637301137 state 0x2 ref 0 00000001:00080000:3.0:1789760725.216150:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972378 is committed 00000100:00100000:0.0:1789760725.226504:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760725.226906:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645312 00000100:00100000:3.0:1789760725.226917:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645312 00000100:00100000:3.0:1789760725.226968:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703572535166 time 703637301137 state 0x0 ref 1 00000100:00100000:3.0:1789760725.226983:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703637561553 time 703637301137 state 0x1 ref 1 00000100:00100000:3.0:1789760725.227015:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703637561553 time 703692303733 state 0x0 ref 1 00000100:00100000:3.0:1789760725.227036:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515645312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760725.227082:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20733952, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:3.0:1789760725.227100:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.227124:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760725.227128:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.227135:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.227305:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.288884:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972379 is committed 00002000:00010000:3.0:1789760725.288949:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20733952 -> 20738048 00010000:00010000:3.0:1789760725.288963:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.288976:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.288986:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.288993:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760725.288998:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.289006:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.289025:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.289033:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dcd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.289228:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515645312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62198us (62739us total) trans 4294972379 rc 0/0 00000100:00100000:3.0:1789760725.289240:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5091 00000100:00100000:3.0:1789760725.289249:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703637561553 time 703692303733 state 0x2 ref 0 00000100:00100000:0.0:1789760725.299602:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760725.299810:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645440 00000100:00100000:1.0:1789760725.299820:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645440 00000100:00100000:1.0:1789760725.299873:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703637561553 time 703692303733 state 0x0 ref 1 00000100:00100000:1.0:1789760725.299889:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703692564149 time 703692303733 state 0x1 ref 1 00000100:00100000:1.0:1789760725.299922:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703692564149 time 703765210638 state 0x0 ref 1 00000100:00100000:1.0:1789760725.299935:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515645440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760725.299986:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20738048, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:1.0:1789760725.299995:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.300011:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760725.300016:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.300022:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.300036:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.309865:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760725.366431:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20738048 -> 20742144 00010000:00010000:3.0:1789760725.366733:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.366748:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.366758:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.366765:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760725.366770:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.366779:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.366798:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.366806:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dcdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.366905:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515645440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66968us (67308us total) trans 4294972380 rc 0/0 00000100:00100000:3.0:1789760725.366915:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5092 00000100:00100000:3.0:1789760725.366924:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703692564149 time 703765210638 state 0x2 ref 0 00000001:00080000:3.0:1789760725.366968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972380 is committed 00000100:00100000:1.0:1789760725.370521:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760725.370611:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645568 00000100:00100000:3.0:1789760725.370621:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645568 00000100:00100000:3.0:1789760725.370686:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703692564149 time 703765210638 state 0x0 ref 1 00000100:00100000:3.0:1789760725.370703:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703765471054 time 703765210638 state 0x1 ref 1 00000100:00100000:3.0:1789760725.370764:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703765471054 time 703836052713 state 0x0 ref 1 00000100:00100000:3.0:1789760725.370777:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515645568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760725.370825:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20742144, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:3.0:1789760725.370834:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.370851:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760725.370856:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.370862:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.370875:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.447072:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972381 is committed 00002000:00010000:2.0:1789760725.447257:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20742144 -> 20746240 00010000:00010000:2.0:1789760725.447273:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447285:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447298:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447306:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760725.447311:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447319:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447338:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447346:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1dce2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.447433:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515645568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 76654us (76918us total) trans 4294972381 rc 0/0 00000100:00100000:2.0:1789760725.447444:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5093 00000100:00100000:2.0:1789760725.447454:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703765471054 time 703836052713 state 0x2 ref 0 00000100:00100000:0.0:1789760725.451747:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760725.451845:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645696 00000100:00100000:1.0:1789760725.451856:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645696 00000100:00100000:1.0:1789760725.451908:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703765471054 time 703836052713 state 0x0 ref 1 00000100:00100000:1.0:1789760725.451925:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703836313129 time 703836052713 state 0x1 ref 1 00000100:00100000:1.0:1789760725.451957:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703836313129 time 703917246136 state 0x0 ref 1 00000100:00100000:1.0:1789760725.451970:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515645696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760725.452017:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20746240, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:1.0:1789760725.452263:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.452279:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760725.452284:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.452290:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.452305:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.483424:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20746240 -> 20750336 00010000:00010000:0.0:1789760725.483443:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.483457:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.483465:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.483479:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760725.483484:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.483492:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.483511:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.483519:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dce9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.483642:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972382 is committed 00000100:00100000:0.0:1789760725.483690:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515645696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31717us (31948us total) trans 4294972382 rc 0/0 00000100:00100000:0.0:1789760725.483700:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5094 00000100:00100000:0.0:1789760725.483743:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703836313129 time 703917246136 state 0x2 ref 0 00000100:00100000:2.0:1789760725.488998:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760725.489095:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645824 00000100:00100000:0.0:1789760725.489229:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645824 00000100:00100000:0.0:1789760725.489282:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703836313129 time 703917246136 state 0x0 ref 1 00000100:00100000:0.0:1789760725.489297:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703917506552 time 703917246136 state 0x1 ref 1 00000100:00100000:0.0:1789760725.489368:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703917506552 time 703954656797 state 0x0 ref 1 00000100:00100000:0.0:1789760725.489389:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515645824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760725.489435:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20750336, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:0.0:1789760725.489443:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.489457:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760725.489462:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.489468:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.489481:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.523959:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972383 is committed 00002000:00010000:2.0:1789760725.525313:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20750336 -> 20754432 00010000:00010000:2.0:1789760725.525330:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.525348:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.525356:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.525362:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760725.525367:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.525376:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.525396:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.525404:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.525672:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515645824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36279us (36679us total) trans 4294972383 rc 0/0 00000100:00100000:2.0:1789760725.525683:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5095 00000100:00100000:2.0:1789760725.525693:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703917506552 time 703954656797 state 0x2 ref 0 00000100:00100000:1.0:1789760725.529346:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760725.529425:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515645952 00000100:00100000:2.0:1789760725.529436:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515645952 00000100:00100000:2.0:1789760725.529498:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703917506552 time 703954656797 state 0x0 ref 1 00000100:00100000:2.0:1789760725.529514:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703954917213 time 703954656797 state 0x1 ref 1 00000100:00100000:2.0:1789760725.529543:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703954917213 time 703994831946 state 0x0 ref 1 00000100:00100000:2.0:1789760725.529556:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515645952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760725.529605:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20754432, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:2.0:1789760725.529615:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.529631:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760725.529636:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.529642:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.529656:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760725.560395:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20754432 -> 20758528 00010000:00010000:2.0:1789760725.560415:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.560428:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.560442:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.560450:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760725.560454:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.560536:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.560560:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.560570:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1dcf7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.560668:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515645952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31110us (31326us total) trans 4294972384 rc 0/0 00000100:00100000:2.0:1789760725.560678:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5096 00000100:00100000:2.0:1789760725.560688:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703954917213 time 703994831946 state 0x2 ref 0 00000001:00080000:0.0:1789760725.562190:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972384 is committed 00000100:00100000:3.0:1789760725.564855:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760725.564962:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646080 00000100:00100000:0.0:1789760725.564972:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646080 00000100:00100000:0.0:1789760725.565032:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703954917213 time 703994831946 state 0x0 ref 1 00000100:00100000:0.0:1789760725.565046:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703995092362 time 703994831946 state 0x1 ref 1 00000100:00100000:0.0:1789760725.565076:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703995092362 time 704030365702 state 0x0 ref 1 00000100:00100000:0.0:1789760725.565088:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515646080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760725.565143:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20758528, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:0.0:1789760725.565153:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.565166:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.565170:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.565176:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.565187:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.603924:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20758528 -> 20762624 00010000:00010000:1.0:1789760725.603943:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.603956:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.603964:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.603970:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760725.603974:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.603983:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.604001:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.604008:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1dcfe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.604123:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515646080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39027us (39268us total) trans 4294972385 rc 0/0 00000100:00100000:1.0:1789760725.604135:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5097 00000100:00100000:1.0:1789760725.604145:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703995092362 time 704030365702 state 0x2 ref 0 00000001:00080000:0.0:1789760725.605812:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972385 is committed 00000100:00100000:1.0:1789760725.614630:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760725.614967:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646208 00000100:00100000:0.0:1789760725.614975:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646208 00000100:00100000:0.0:1789760725.615032:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 703995092362 time 704030365702 state 0x0 ref 1 00000100:00100000:0.0:1789760725.615047:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704030626118 time 704030365702 state 0x1 ref 1 00000100:00100000:0.0:1789760725.615076:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704030626118 time 704080366139 state 0x0 ref 1 00000100:00100000:0.0:1789760725.615087:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515646208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760725.615143:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20762624, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:0.0:1789760725.615152:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.615167:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.615171:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.615178:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.615191:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.659699:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972386 is committed 00002000:00010000:2.0:1789760725.659932:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20762624 -> 20766720 00010000:00010000:2.0:1789760725.659947:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.659964:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.659973:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.659979:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760725.659984:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.659994:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.660014:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.660023:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1dd05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.660118:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515646208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45011us (45476us total) trans 4294972386 rc 0/0 00000100:00100000:2.0:1789760725.660129:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5098 00000100:00100000:2.0:1789760725.660139:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704030626118 time 704080366139 state 0x2 ref 0 00000100:00100000:3.0:1789760725.666866:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760725.666952:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646336 00000100:00100000:2.0:1789760725.666961:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646336 00000100:00100000:2.0:1789760725.667016:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704030626118 time 704080366139 state 0x0 ref 1 00000100:00100000:2.0:1789760725.667126:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704080626555 time 704080366139 state 0x1 ref 1 00000100:00100000:2.0:1789760725.667158:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704080626555 time 704132446741 state 0x0 ref 1 00000100:00100000:2.0:1789760725.667286:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515646336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760725.667331:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20766720, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:2.0:1789760725.667340:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.667355:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.667359:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.667366:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.667380:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.713438:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20766720 -> 20770816 00010000:00010000:1.0:1789760725.713457:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713483:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713492:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713500:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760725.713505:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713515:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713535:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713544:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1dd0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.713643:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515646336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46355us (46780us total) trans 4294972387 rc 0/0 00000100:00100000:1.0:1789760725.713653:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5099 00000100:00100000:1.0:1789760725.713662:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704080626555 time 704132446741 state 0x2 ref 0 00000001:00080000:2.0:1789760725.714422:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972387 is committed 00000100:00100000:3.0:1789760725.737429:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760725.743574:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646464 00000100:00100000:1.0:1789760725.743590:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646464 00000100:00100000:2.0:1789760725.743923:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704080626555 time 704132446741 state 0x0 ref 1 00000100:00100000:2.0:1789760725.743939:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704132707157 time 704132446741 state 0x1 ref 1 00000100:00100000:2.0:1789760725.745553:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704132707157 time 704210832894 state 0x0 ref 1 00000100:00100000:2.0:1789760725.745584:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515646464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760725.745635:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20770816, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:2.0:1789760725.745644:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.745672:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.745677:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.745684:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.745700:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.799740:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972388 is committed 00002000:00010000:2.0:1789760725.799939:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20770816 -> 20774912 00010000:00010000:2.0:1789760725.799955:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.799967:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.799976:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.799983:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760725.799987:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.799997:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.800015:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.800089:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1dd13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.800291:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515646464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54713us (62866us total) trans 4294972388 rc 0/0 00000100:00100000:2.0:1789760725.800303:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5100 00000100:00100000:2.0:1789760725.800312:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704132707157 time 704210832894 state 0x2 ref 0 00000100:00100000:0.0:1789760725.806253:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760725.811299:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646592 00000100:00100000:2.0:1789760725.811315:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646592 00000100:00100000:3.0:1789760725.814455:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704132707157 time 704210832894 state 0x0 ref 1 00000100:00100000:3.0:1789760725.814551:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704211093310 time 704210832894 state 0x1 ref 1 00000100:00100000:3.0:1789760725.814641:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704211093310 time 704279929038 state 0x0 ref 1 00000100:00100000:3.0:1789760725.814668:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515646592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760725.814748:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20774912, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:3.0:1789760725.814758:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.814845:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.814849:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.814856:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.814869:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.885777:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20774912 -> 20779008 00010000:00010000:0.0:1789760725.885797:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.885809:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.885819:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.885827:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760725.885832:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.885850:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.885870:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.885885:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1dd1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.885976:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515646592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71317us (79729us total) trans 4294972389 rc 0/0 00000100:00100000:0.0:1789760725.885988:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5101 00000100:00100000:0.0:1789760725.885999:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704211093310 time 704279929038 state 0x2 ref 0 00000001:00080000:0.0:1789760725.886040:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972389 is committed 00000100:00100000:3.0:1789760725.890315:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760725.896628:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646720 00000100:00100000:1.0:1789760725.896644:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646720 00000100:00100000:2.0:1789760725.897434:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704211093310 time 704279929038 state 0x0 ref 1 00000100:00100000:2.0:1789760725.897451:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704280189454 time 704279929038 state 0x1 ref 1 00000100:00100000:2.0:1789760725.897626:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704280189454 time 704362912970 state 0x0 ref 1 00000100:00100000:2.0:1789760725.897641:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515646720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760725.897701:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20779008, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xa007 00010000:00010000:2.0:1789760725.897733:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.897750:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.897754:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.897761:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.897775:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.946482:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972390 is committed 00002000:00010000:0.0:1789760725.946982:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20779008 -> 20783104 00002000:00010000:0.0:1789760725.946988:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760725 -> 1789760726 00002000:00010000:0.0:1789760725.946991:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760725 -> 1789760726 00010000:00010000:0.0:1789760725.947006:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.947085:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.947095:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.947118:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760725.947123:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.947138:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.947161:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.947245:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1dd21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.947345:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515646720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49701us (57033us total) trans 4294972390 rc 0/0 00000100:00100000:0.0:1789760725.947355:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5102 00000100:00100000:0.0:1789760725.947364:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704280189454 time 704362912970 state 0x2 ref 0 00000100:00100000:2.0:1789760725.952997:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760725.953325:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646848 00000100:00100000:1.0:1789760725.953336:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646848 00000100:00100000:1.0:1789760725.953390:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704280189454 time 704362912970 state 0x0 ref 1 00000100:00100000:1.0:1789760725.953405:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704363173386 time 704362912970 state 0x1 ref 1 00000100:00100000:1.0:1789760725.953436:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704363173386 time 704418725290 state 0x0 ref 1 00000100:00100000:1.0:1789760725.953450:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515646848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760725.953510:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20783104, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:1.0:1789760725.953519:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.953535:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760725.953540:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.953546:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.953566:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.004565:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972391 is committed 00002000:00010000:2.0:1789760726.004801:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20783104 -> 20787200 00010000:00010000:2.0:1789760726.004816:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.004950:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.004961:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.004968:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760726.004973:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.004983:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.005010:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.005087:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1dd28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.005380:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515646848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51927us (52389us total) trans 4294972391 rc 0/0 00000100:00100000:2.0:1789760726.005393:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5103 00000100:00100000:2.0:1789760726.005404:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704363173386 time 704418725290 state 0x2 ref 0 00000100:00100000:2.0:1789760726.033462:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.034138:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515646976 00000100:00100000:0.0:1789760726.034150:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515646976 00000100:00100000:0.0:1789760726.034229:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704363173386 time 704418725290 state 0x0 ref 1 00000100:00100000:0.0:1789760726.034245:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704418985706 time 704418725290 state 0x1 ref 1 00000100:00080000:2.0:1789760726.034324:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760726.034402:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704418985706 time 704499690479 state 0x0 ref 1 00000100:00100000:0.0:1789760726.034416:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515646976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.034463:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20787200, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.034474:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.034492:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.034497:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.034503:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.034517:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.094272:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972392 is committed 00002000:00010000:0.0:1789760726.095441:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20787200 -> 20791296 00010000:00010000:0.0:1789760726.095966:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.095979:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.095988:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.095995:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.096000:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.096009:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.096259:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.096267:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.096410:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515646976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61992us (62954us total) trans 4294972392 rc 0/0 00000100:00100000:0.0:1789760726.096421:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5104 00000100:00100000:0.0:1789760726.096430:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704418985706 time 704499690479 state 0x2 ref 0 00000100:00100000:3.0:1789760726.101870:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760726.105287:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647104 00000100:00100000:1.0:1789760726.105304:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647104 00000100:00100000:1.0:1789760726.105374:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704418985706 time 704499690479 state 0x0 ref 1 00000100:00100000:1.0:1789760726.105391:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704499950895 time 704499690479 state 0x1 ref 1 00000100:00100000:1.0:1789760726.105418:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704499950895 time 704570707046 state 0x0 ref 1 00000100:00100000:1.0:1789760726.105432:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515647104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760726.105484:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20791296, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:1.0:1789760726.105493:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.105510:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760726.105514:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.105521:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.105537:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.143468:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972393 is committed 00002000:00010000:3.0:1789760726.143663:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20791296 -> 20795392 00010000:00010000:3.0:1789760726.143680:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.143693:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.143703:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.143858:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760726.143863:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.143872:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.143892:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.143900:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1dd36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.144092:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515647104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38656us (42225us total) trans 4294972393 rc 0/0 00000100:00100000:3.0:1789760726.144170:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5105 00000100:00100000:3.0:1789760726.144180:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704499950895 time 704570707046 state 0x2 ref 0 00000100:00100000:2.0:1789760726.148382:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.148479:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647232 00000100:00100000:0.0:1789760726.148490:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647232 00000100:00100000:0.0:1789760726.148544:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704499950895 time 704570707046 state 0x0 ref 1 00000100:00100000:0.0:1789760726.148560:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704570967462 time 704570707046 state 0x1 ref 1 00000100:00100000:0.0:1789760726.148653:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704570967462 time 704613941467 state 0x0 ref 1 00000100:00100000:0.0:1789760726.148667:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515647232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.149663:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20795392, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.149676:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.149691:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.149696:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.149703:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.149872:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.184364:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972394 is committed 00002000:00010000:0.0:1789760726.184469:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20795392 -> 20799488 00010000:00010000:0.0:1789760726.184485:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.184498:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.184507:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.184515:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.184520:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.184530:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.184549:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.184557:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.184661:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515647232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35991us (36283us total) trans 4294972394 rc 0/0 00000100:00100000:0.0:1789760726.184672:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5106 00000100:00100000:0.0:1789760726.184683:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704570967462 time 704613941467 state 0x2 ref 0 00000100:00100000:1.0:1789760726.189066:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760726.189140:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647360 00000100:00100000:3.0:1789760726.189150:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647360 00000100:00100000:3.0:1789760726.189204:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704570967462 time 704613941467 state 0x0 ref 1 00000100:00100000:3.0:1789760726.189220:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704614201883 time 704613941467 state 0x1 ref 1 00000100:00100000:3.0:1789760726.189254:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704614201883 time 704654542864 state 0x0 ref 1 00000100:00100000:3.0:1789760726.189266:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515647360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760726.189314:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20799488, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:3.0:1789760726.189324:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.189342:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760726.189346:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.189353:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.189368:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.277773:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20799488 -> 20803584 00010000:00010000:3.0:1789760726.277797:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.277810:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.277819:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.277826:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760726.277832:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.277841:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.277860:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.277868:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1dd44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.278077:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972395 is committed 00000100:00100000:3.0:1789760726.278200:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515647360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 88931us (89140us total) trans 4294972395 rc 0/0 00000100:00100000:3.0:1789760726.278211:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5107 00000100:00100000:3.0:1789760726.278220:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704614201883 time 704654542864 state 0x2 ref 0 00000100:00100000:1.0:1789760726.282546:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.282950:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647488 00000100:00100000:0.0:1789760726.282961:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647488 00000100:00100000:0.0:1789760726.283118:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704614201883 time 704654542864 state 0x0 ref 1 00000100:00100000:0.0:1789760726.283137:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704654803280 time 704654542864 state 0x1 ref 1 00000100:00100000:0.0:1789760726.283168:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704654803280 time 704748456985 state 0x0 ref 1 00000100:00100000:0.0:1789760726.283181:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515647488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.283229:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20803584, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.283239:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.283255:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.283261:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.283268:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.283282:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.309877:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20803584 -> 20807680 00010000:00010000:3.0:1789760726.309906:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.309922:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.309932:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.309946:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760726.309952:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.309962:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.309982:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.309990:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1dd4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.310103:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515647488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26910us (27552us total) trans 4294972396 rc 0/0 00000100:00100000:3.0:1789760726.310116:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5108 00000100:00100000:3.0:1789760726.310126:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704654803280 time 704748456985 state 0x2 ref 0 00000100:00100000:0.0:1789760726.313809:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760726.313891:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972396 is committed 00000100:00100000:1.0:1789760726.313945:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647616 00000100:00100000:1.0:1789760726.313956:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647616 00000100:00100000:1.0:1789760726.314011:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704654803280 time 704748456985 state 0x0 ref 1 00000100:00100000:1.0:1789760726.314031:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704748717401 time 704748456985 state 0x1 ref 1 00000100:00100000:1.0:1789760726.314061:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704748717401 time 704779349946 state 0x0 ref 1 00000100:00100000:1.0:1789760726.314074:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515647616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760726.314130:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20807680, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:1.0:1789760726.314139:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.314155:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760726.314159:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.314166:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.314180:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.334120:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760726.338567:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20807680 -> 20811776 00010000:00010000:1.0:1789760726.338589:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338603:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338612:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338619:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760726.338625:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338634:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338656:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.338663:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1dd52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.338791:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515647616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24714us (24986us total) trans 4294972397 rc 0/0 00000100:00100000:1.0:1789760726.338801:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5109 00000100:00100000:1.0:1789760726.338810:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704748717401 time 704779349946 state 0x2 ref 0 00000001:00080000:2.0:1789760726.339483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972397 is committed 00000100:00100000:1.0:1789760726.343311:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.343362:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647744 00000100:00100000:0.0:1789760726.343373:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647744 00000100:00100000:0.0:1789760726.343429:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704748717401 time 704779349946 state 0x0 ref 1 00000100:00100000:0.0:1789760726.343444:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704779610362 time 704779349946 state 0x1 ref 1 00000100:00100000:0.0:1789760726.343472:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704779610362 time 704808761006 state 0x0 ref 1 00000100:00100000:0.0:1789760726.343485:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515647744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.343533:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20811776, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.343542:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.343558:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.343563:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.343569:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.343583:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.388897:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972398 is committed 00002000:00010000:0.0:1789760726.389084:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20811776 -> 20815872 00010000:00010000:0.0:1789760726.389114:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389129:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389139:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389147:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.389152:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389162:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389183:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.389191:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.389330:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515647744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45842us (46022us total) trans 4294972398 rc 0/0 00000100:00100000:0.0:1789760726.389341:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5110 00000100:00100000:0.0:1789760726.389350:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704779610362 time 704808761006 state 0x2 ref 0 00000100:00100000:3.0:1789760726.393438:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.393531:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515647872 00000100:00100000:0.0:1789760726.393543:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515647872 00000100:00100000:0.0:1789760726.393602:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704779610362 time 704808761006 state 0x0 ref 1 00000100:00100000:0.0:1789760726.393618:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704809021422 time 704808761006 state 0x1 ref 1 00000100:00100000:0.0:1789760726.393652:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704809021422 time 704858941084 state 0x0 ref 1 00000100:00100000:0.0:1789760726.393666:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515647872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.393879:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20815872, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.393889:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.393906:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.393910:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.393917:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.393931:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.422467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972399 is committed 00002000:00010000:0.0:1789760726.422554:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20815872 -> 20819968 00010000:00010000:0.0:1789760726.422570:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.422583:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.422598:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.422605:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.422611:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.422621:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.422640:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.422648:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.422890:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515647872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29221us (29456us total) trans 4294972399 rc 0/0 00000100:00100000:0.0:1789760726.422902:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5111 00000100:00100000:0.0:1789760726.422911:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704809021422 time 704858941084 state 0x2 ref 0 00000100:00100000:1.0:1789760726.427232:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.427399:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648000 00000100:00100000:0.0:1789760726.427410:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648000 00000100:00100000:0.0:1789760726.427464:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704809021422 time 704858941084 state 0x0 ref 1 00000100:00100000:0.0:1789760726.427481:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704859201500 time 704858941084 state 0x1 ref 1 00000100:00100000:0.0:1789760726.427545:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704859201500 time 704892834181 state 0x0 ref 1 00000100:00100000:0.0:1789760726.427559:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515648000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.427607:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20819968, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.427615:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.427630:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.427634:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.427640:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.427654:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.453618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972400 is committed 00002000:00010000:2.0:1789760726.454124:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20819968 -> 20824064 00010000:00010000:2.0:1789760726.454142:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.454156:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.454172:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.454180:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760726.454186:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.454195:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.454215:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.454224:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1dd67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.454368:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515648000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26806us (27141us total) trans 4294972400 rc 0/0 00000100:00100000:2.0:1789760726.454379:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5112 00000100:00100000:2.0:1789760726.454389:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704859201500 time 704892834181 state 0x2 ref 0 00000100:00100000:3.0:1789760726.457973:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.458159:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648128 00000100:00100000:0.0:1789760726.458169:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648128 00000100:00100000:0.0:1789760726.458228:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704859201500 time 704892834181 state 0x0 ref 1 00000100:00100000:0.0:1789760726.458245:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704893094597 time 704892834181 state 0x1 ref 1 00000100:00100000:0.0:1789760726.458280:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704893094597 time 704923569289 state 0x0 ref 1 00000100:00100000:0.0:1789760726.458292:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515648128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.458346:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20824064, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.458356:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.458373:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760726.458377:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.458383:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.458397:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.492835:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972401 is committed 00002000:00010000:2.0:1789760726.492953:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20824064 -> 20828160 00010000:00010000:2.0:1789760726.492971:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.492990:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.493000:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.493007:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760726.493020:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.493030:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.493052:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.493061:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1dd6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.493177:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515648128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34880us (35209us total) trans 4294972401 rc 0/0 00000100:00100000:2.0:1789760726.493189:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5113 00000100:00100000:2.0:1789760726.493200:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704893094597 time 704923569289 state 0x2 ref 0 00000100:00100000:1.0:1789760726.500178:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760726.500272:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648256 00000100:00100000:2.0:1789760726.500283:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648256 00000100:00100000:2.0:1789760726.500351:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704893094597 time 704923569289 state 0x0 ref 1 00000100:00100000:2.0:1789760726.500370:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704923829705 time 704923569289 state 0x1 ref 1 00000100:00100000:2.0:1789760726.500450:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704923829705 time 704965738866 state 0x0 ref 1 00000100:00100000:2.0:1789760726.500463:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515648256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760726.500518:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20828160, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:2.0:1789760726.500527:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.500543:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760726.500548:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.500554:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.500568:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.527649:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544408832:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.527765:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760726.527960:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544409216:0@lo:400:kworker.0 00000100:00100000:2.0:1789760726.527994:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760726.528561:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8029c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544409344:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.528594:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760726.528784:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408832 00000100:00100000:3.0:1789760726.528792:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408832 00000100:00100000:3.0:1789760726.528879:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:3.0:1789760726.528888:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909351450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544408832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.528946:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909351450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544408832:12345-0@lo:400:kworker.0 Request processed in 59us (1181us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760726.528953:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789760726.529011:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409344 00000100:00100000:1.0:1789760726.529016:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409344 00000100:00100000:1.0:1789760726.529058:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:1.0:1789760726.529065:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9093daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544409344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760726.529113:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9093daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544409344:12345-0@lo:400:kworker.0 Request processed in 49us (518us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760726.529120:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1789760726.529216:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409216 00000100:00100000:2.0:1789760726.529221:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409216 00000100:00100000:2.0:1789760726.529257:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789760726.529265:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544409216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760726.529300:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544409216:12345-0@lo:400:kworker.0 Request processed in 37us (1305us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760726.529307:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00080000:2.0:1789760726.529323:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760726.529327:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760726.529355:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544409216:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532299:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544409088:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532373:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760726.532424:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789760726.532431:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544408960:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532462:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760726.532580:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409088 00000100:00100000:0.0:1789760726.532589:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409088 00000100:00100000:3.0:1789760726.532631:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8029c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544409472:0@lo:400:kworker.0 00000100:00100000:0.0:1789760726.532640:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544408960 00000100:00100000:0.0:1789760726.532645:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544408960 00000100:00100000:3.0:1789760726.532662:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760726.532674:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1789760726.532685:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae9093da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544409088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532704:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544408832:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532769:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8029c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544409344:0@lo:400:kworker.0 00000100:00100000:0.0:1789760726.532774:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae9093da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544409088:12345-0@lo:400:kworker.0 Request processed in 88us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760726.532784:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1789760726.532805:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1789760726.532814:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544408960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532822:0:8226:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409472 00000100:00100000:3.0:1789760726.532827:0:8226:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409472 00000100:00100000:0.0:1789760726.532851:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544408960:12345-0@lo:400:kworker.0 Request processed in 38us (386us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760726.532859:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1789760726.532864:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:3.0:1789760726.532873:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544409472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.532911:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544409472:12345-0@lo:400:kworker.0 Request processed in 40us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760726.532919:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00080000:2.0:1789760726.532924:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760726.532931:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760726.533155:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544408960:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.533231:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8029c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544409472:0@lo:400:kworker.0 00000100:00100000:3.0:1789760726.533245:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544409088:0@lo:400:kworker.0 00002000:00010000:2.0:1789760726.561429:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20828160 -> 20832256 00010000:00010000:2.0:1789760726.561452:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.561466:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.561475:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.561483:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760726.561487:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.561501:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.561526:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.561533:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dd75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.561641:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515648256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61173us (61465us total) trans 4294972402 rc 0/0 00000001:00080000:0.0:1789760726.561645:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972402 is committed 00000100:00100000:2.0:1789760726.561654:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5114 00000100:00100000:2.0:1789760726.561666:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704923829705 time 704965738866 state 0x2 ref 0 00000100:00100000:3.0:1789760726.565666:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760726.565772:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648384 00000100:00100000:2.0:1789760726.565783:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648384 00000100:00100000:2.0:1789760726.565839:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704923829705 time 704965738866 state 0x0 ref 1 00000100:00100000:2.0:1789760726.565855:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704965999282 time 704965738866 state 0x1 ref 1 00000100:00100000:2.0:1789760726.565888:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704965999282 time 705031177730 state 0x0 ref 1 00000100:00100000:2.0:1789760726.565900:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515648384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760726.565955:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20832256, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:2.0:1789760726.565964:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.565981:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760726.565985:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.565991:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.566005:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.625330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972403 is committed 00002000:00010000:2.0:1789760726.625420:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20832256 -> 20836352 00010000:00010000:2.0:1789760726.625438:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.625451:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.625460:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.625467:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760726.625473:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.625483:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.625505:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.625513:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.625665:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515648384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59760us (60003us total) trans 4294972403 rc 0/0 00000100:00100000:2.0:1789760726.625679:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5115 00000100:00100000:2.0:1789760726.625689:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704965999282 time 705031177730 state 0x2 ref 0 00000100:00100000:3.0:1789760726.628689:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760726.628809:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648512 00000100:00100000:1.0:1789760726.628819:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648512 00000100:00100000:1.0:1789760726.628874:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 704965999282 time 705031177730 state 0x0 ref 1 00000100:00100000:1.0:1789760726.628889:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705031438146 time 705031177730 state 0x1 ref 1 00000100:00100000:1.0:1789760726.628920:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705031438146 time 705094209594 state 0x0 ref 1 00000100:00100000:1.0:1789760726.628933:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515648512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760726.628983:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20836352, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:1.0:1789760726.628992:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.629008:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.629013:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.629019:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.629117:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.653937:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972404 is committed 00002000:00010000:0.0:1789760726.654052:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20836352 -> 20840448 00010000:00010000:0.0:1789760726.654068:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.654081:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.654209:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.654220:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.654226:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.654236:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.654255:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.654264:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1dd83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.654360:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515648512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25423us (25676us total) trans 4294972404 rc 0/0 00000100:00100000:0.0:1789760726.654371:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5116 00000100:00100000:0.0:1789760726.654381:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705031438146 time 705094209594 state 0x2 ref 0 00000100:00100000:1.0:1789760726.659624:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.661001:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648640 00000100:00100000:0.0:1789760726.661015:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648640 00000100:00100000:0.0:1789760726.661072:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705031438146 time 705094209594 state 0x0 ref 1 00000100:00100000:0.0:1789760726.661271:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705094470010 time 705094209594 state 0x1 ref 1 00000100:00100000:0.0:1789760726.661352:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705094470010 time 705126640643 state 0x0 ref 1 00000100:00100000:0.0:1789760726.661367:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515648640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.661494:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20840448, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.661503:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.661519:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.661524:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.661531:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.661546:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.688142:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972405 is committed 00002000:00010000:0.0:1789760726.688484:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20840448 -> 20844544 00010000:00010000:0.0:1789760726.688499:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.688513:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.688523:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.688530:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.688535:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.688544:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.688562:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.688570:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.688690:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515648640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27320us (29071us total) trans 4294972405 rc 0/0 00000100:00100000:0.0:1789760726.688700:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5117 00000100:00100000:0.0:1789760726.688736:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705094470010 time 705126640643 state 0x2 ref 0 00000100:00100000:3.0:1789760726.693174:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.693266:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648768 00000100:00100000:0.0:1789760726.693277:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648768 00000100:00100000:0.0:1789760726.693329:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705094470010 time 705126640643 state 0x0 ref 1 00000100:00100000:0.0:1789760726.693345:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705126901059 time 705126640643 state 0x1 ref 1 00000100:00100000:0.0:1789760726.693472:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705126901059 time 705158759496 state 0x0 ref 1 00000100:00100000:0.0:1789760726.693485:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515648768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.693528:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20844544, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.693536:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.693552:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.693556:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.693562:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.693577:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.728025:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972406 is committed 00002000:00010000:0.0:1789760726.728252:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20844544 -> 20848640 00010000:00010000:0.0:1789760726.728267:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.728280:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.728290:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.728298:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.728304:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.728313:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.728334:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.728409:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dd91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.728562:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515648768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35073us (35392us total) trans 4294972406 rc 0/0 00000100:00100000:0.0:1789760726.728574:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5118 00000100:00100000:0.0:1789760726.728584:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705126901059 time 705158759496 state 0x2 ref 0 00000100:00100000:2.0:1789760726.734376:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.734469:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515648896 00000100:00100000:0.0:1789760726.734478:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515648896 00000100:00100000:0.0:1789760726.734529:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705126901059 time 705158759496 state 0x0 ref 1 00000100:00100000:0.0:1789760726.734543:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705159019912 time 705158759496 state 0x1 ref 1 00000100:00100000:0.0:1789760726.734575:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705159019912 time 705199864069 state 0x0 ref 1 00000100:00100000:0.0:1789760726.734586:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515648896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760726.734638:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20848640, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:0.0:1789760726.734648:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.734665:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.734670:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.734676:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.734689:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760726.784605:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20848640 -> 20852736 00010000:00010000:0.0:1789760726.784623:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.784637:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.784646:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.784653:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760726.784657:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.784666:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.784689:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.784697:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dd98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.784799:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515648896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50209us (50428us total) trans 4294972407 rc 0/0 00000100:00100000:0.0:1789760726.784809:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5119 00000100:00100000:0.0:1789760726.784817:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705159019912 time 705199864069 state 0x2 ref 0 00000001:00080000:2.0:1789760726.784877:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972407 is committed 00000100:00100000:1.0:1789760726.788976:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760726.789129:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649024 00000100:00100000:2.0:1789760726.789139:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649024 00000100:00100000:2.0:1789760726.789192:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705159019912 time 705199864069 state 0x0 ref 1 00000100:00100000:2.0:1789760726.789207:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705200124485 time 705199864069 state 0x1 ref 1 00000100:00100000:2.0:1789760726.789268:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705200124485 time 705254556377 state 0x0 ref 1 00000100:00100000:2.0:1789760726.789281:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515649024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760726.789326:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20852736, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:2.0:1789760726.789374:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.789392:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760726.789397:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.789403:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.789417:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.842664:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972408 is committed 00002000:00010000:1.0:1789760726.842957:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20852736 -> 20856832 00010000:00010000:1.0:1789760726.842971:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842984:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.842993:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.843001:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760726.843006:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.843080:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.843108:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.843120:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1dd9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.843303:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515649024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54019us (54331us total) trans 4294972408 rc 0/0 00000100:00100000:1.0:1789760726.843314:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5120 00000100:00100000:1.0:1789760726.843324:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705200124485 time 705254556377 state 0x2 ref 0 00000100:00100000:3.0:1789760726.848671:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760726.849066:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649152 00000100:00100000:2.0:1789760726.849076:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649152 00000100:00100000:2.0:1789760726.849275:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705200124485 time 705254556377 state 0x0 ref 1 00000100:00100000:2.0:1789760726.849290:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705254816793 time 705254556377 state 0x1 ref 1 00000100:00100000:2.0:1789760726.849321:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705254816793 time 705314610857 state 0x0 ref 1 00000100:00100000:2.0:1789760726.849334:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515649152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760726.850261:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20856832, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xa007 00010000:00010000:2.0:1789760726.850274:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.850289:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760726.850294:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.850300:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.850314:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.927739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972409 is committed 00002000:00010000:1.0:1789760726.928267:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20856832 -> 20860928 00002000:00010000:1.0:1789760726.928272:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760726 -> 1789760727 00002000:00010000:1.0:1789760726.928274:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760726 -> 1789760727 00010000:00010000:1.0:1789760726.928289:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.928302:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.928311:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.928318:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760726.928322:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.928331:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.928353:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.928362:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dda6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.928571:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515649152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 79235us (79905us total) trans 4294972409 rc 0/0 00000100:00100000:1.0:1789760726.928582:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5121 00000100:00100000:1.0:1789760726.928592:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705254816793 time 705314610857 state 0x2 ref 0 00000100:00100000:2.0:1789760726.936423:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760726.938356:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649280 00000100:00100000:1.0:1789760726.938370:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649280 00000100:00100000:1.0:1789760726.938429:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705254816793 time 705314610857 state 0x0 ref 1 00000100:00100000:1.0:1789760726.938456:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705314871273 time 705314610857 state 0x1 ref 1 00000100:00100000:1.0:1789760726.938547:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705314871273 time 705403834578 state 0x0 ref 1 00000100:00100000:1.0:1789760726.938561:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515649280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760726.938613:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20860928, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:1.0:1789760726.938622:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.938638:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.938643:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.938650:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.938664:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.966498:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760726.966597:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649408 00000100:00100000:1.0:1789760726.966616:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649408 00000100:00100000:1.0:1789760726.966673:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 386 00000100:00100000:1.0:1789760726.966683:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909352050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515649408:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760726.966788:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909352050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515649408:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 105us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760726.966795:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 386 00000100:00100000:0.0:1789760726.967195:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760726.967883:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649536 00000100:00100000:1.0:1789760726.967891:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649536 00000100:00080000:2.0:1789760726.967918:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760726.967944:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 441 00000100:00100000:1.0:1789760726.967952:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515649536:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760726.968008:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515649536:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 57us (814us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760726.968021:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 441 00000100:00100000:1.0:1789760726.997305:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760726.999271:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649664 00000100:00100000:0.0:1789760726.999284:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649664 00000100:00100000:0.0:1789760726.999361:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 442 00000100:00100000:0.0:1789760726.999372:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515649664:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760726.999450:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515649664:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 80us (2155us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760726.999458:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 442 00000100:00080000:1.0:1789760726.999899:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:0.0:1789760727.015775:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20860928 -> 20865024 00010000:00010000:0.0:1789760727.015796:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.015814:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.015828:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.015836:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.015840:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.015849:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.015867:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.015875:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ddad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.015976:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515649280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 77413us (79559us total) trans 4294972410 rc 0/0 00000100:00100000:0.0:1789760727.015987:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5122 00000100:00100000:0.0:1789760727.015996:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705314871273 time 705403834578 state 0x2 ref 0 00000001:00080000:3.0:1789760727.016639:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972410 is committed 00000100:00100000:1.0:1789760727.027017:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.027123:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649792 00000100:00100000:0.0:1789760727.027135:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649792 00000100:00100000:0.0:1789760727.027188:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705314871273 time 705403834578 state 0x0 ref 1 00000100:00100000:0.0:1789760727.027204:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705404094994 time 705403834578 state 0x1 ref 1 00000100:00100000:0.0:1789760727.027271:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705404094994 time 705492559683 state 0x0 ref 1 00000100:00100000:0.0:1789760727.027284:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515649792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.027338:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20865024, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:0.0:1789760727.027348:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.027365:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760727.027370:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.027377:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.027393:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.100467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972411 is committed 00002000:00010000:0.0:1789760727.104079:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20865024 -> 20869120 00010000:00010000:0.0:1789760727.104113:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.104125:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.104133:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.104140:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.104145:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.104154:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.104173:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.104179:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.104383:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515649792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 77096us (77371us total) trans 4294972411 rc 0/0 00000100:00100000:0.0:1789760727.104395:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5123 00000100:00100000:0.0:1789760727.104404:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705404094994 time 705492559683 state 0x2 ref 0 00000100:00100000:1.0:1789760727.112582:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.112674:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515649920 00000100:00100000:0.0:1789760727.112684:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515649920 00000100:00100000:0.0:1789760727.112869:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705404094994 time 705492559683 state 0x0 ref 1 00000100:00100000:0.0:1789760727.112886:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705492820099 time 705492559683 state 0x1 ref 1 00000100:00100000:0.0:1789760727.112915:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705492820099 time 705578204988 state 0x0 ref 1 00000100:00100000:0.0:1789760727.112927:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515649920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.112972:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20869120, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:0.0:1789760727.112981:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.112997:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760727.113001:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.113007:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.113020:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.219736:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972412 is committed 00002000:00010000:0.0:1789760727.220049:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20869120 -> 20873216 00010000:00010000:0.0:1789760727.220064:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.220077:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.220086:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.220102:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.220107:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.220117:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.220141:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.220149:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.220243:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515649920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 107312us (107665us total) trans 4294972412 rc 0/0 00000100:00100000:0.0:1789760727.220254:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5124 00000100:00100000:0.0:1789760727.220266:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705492820099 time 705578204988 state 0x2 ref 0 00000100:00100000:2.0:1789760727.227184:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.227275:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650048 00000100:00100000:0.0:1789760727.227285:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650048 00000100:00100000:0.0:1789760727.227338:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705492820099 time 705578204988 state 0x0 ref 1 00000100:00100000:0.0:1789760727.227354:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705578465404 time 705578204988 state 0x1 ref 1 00000100:00100000:0.0:1789760727.227467:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705578465404 time 705692755123 state 0x0 ref 1 00000100:00100000:0.0:1789760727.227481:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515650048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.227530:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20873216, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:0.0:1789760727.227539:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.227555:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760727.227560:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.227567:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.227581:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.313557:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972413 is committed 00002000:00010000:0.0:1789760727.313623:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20873216 -> 20877312 00010000:00010000:0.0:1789760727.313638:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.313652:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.313661:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.313668:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.313673:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.313682:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.313701:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.313742:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ddc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.313826:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515650048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 86342us (86647us total) trans 4294972413 rc 0/0 00000100:00100000:0.0:1789760727.313836:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5125 00000100:00100000:0.0:1789760727.313846:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705578465404 time 705692755123 state 0x2 ref 0 00000100:00100000:1.0:1789760727.324325:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760727.324460:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650176 00000100:00100000:2.0:1789760727.324470:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650176 00000100:00100000:2.0:1789760727.324524:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705578465404 time 705692755123 state 0x0 ref 1 00000100:00100000:2.0:1789760727.324540:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705693015539 time 705692755123 state 0x1 ref 1 00000100:00100000:2.0:1789760727.324569:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705693015539 time 705789858496 state 0x0 ref 1 00000100:00100000:2.0:1789760727.324581:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515650176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760727.324634:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20877312, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:2.0:1789760727.324643:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.324659:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760727.324663:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.324668:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.324682:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.359598:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760727.378373:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972414 is committed 00002000:00010000:2.0:1789760727.380163:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20877312 -> 20881408 00010000:00010000:2.0:1789760727.380180:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.380193:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.380202:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.380209:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760727.380214:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.380223:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.380242:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.380251:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ddc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.380400:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515650176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55815us (56080us total) trans 4294972414 rc 0/0 00000100:00100000:2.0:1789760727.380411:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5126 00000100:00100000:2.0:1789760727.380421:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705693015539 time 705789858496 state 0x2 ref 0 00000100:00100000:0.0:1789760727.386447:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760727.387149:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650304 00000100:00100000:1.0:1789760727.387159:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650304 00000100:00100000:1.0:1789760727.387210:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705693015539 time 705789858496 state 0x0 ref 1 00000100:00100000:1.0:1789760727.387224:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705790118912 time 705789858496 state 0x1 ref 1 00000100:00100000:1.0:1789760727.387291:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705790118912 time 705852579873 state 0x0 ref 1 00000100:00100000:1.0:1789760727.387304:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515650304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760727.387350:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20881408, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:1.0:1789760727.387359:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.387374:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760727.387378:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.387384:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.387397:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.439269:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972415 is committed 00002000:00010000:2.0:1789760727.439470:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20881408 -> 20885504 00010000:00010000:2.0:1789760727.439485:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.439505:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.439514:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.439521:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760727.439526:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.439535:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.439555:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.439563:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ddd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.439683:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515650304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52375us (53240us total) trans 4294972415 rc 0/0 00000100:00100000:2.0:1789760727.439694:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5127 00000100:00100000:2.0:1789760727.439704:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705790118912 time 705852579873 state 0x2 ref 0 00000100:00100000:0.0:1789760727.443210:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760727.443282:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650432 00000100:00100000:1.0:1789760727.443290:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650432 00000100:00100000:1.0:1789760727.443340:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705790118912 time 705852579873 state 0x0 ref 1 00000100:00100000:1.0:1789760727.443355:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705852840289 time 705852579873 state 0x1 ref 1 00000100:00100000:1.0:1789760727.443450:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705852840289 time 705908738259 state 0x0 ref 1 00000100:00100000:1.0:1789760727.443463:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515650432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760727.443512:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20885504, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:1.0:1789760727.443521:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.443537:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760727.443542:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.443548:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.443562:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.470494:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20885504 -> 20889600 00010000:00010000:1.0:1789760727.470514:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.470526:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.470534:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.470541:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760727.470546:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.470555:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.470574:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.470582:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ddd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.470678:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515650432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27209us (27468us total) trans 4294972416 rc 0/0 00000100:00100000:1.0:1789760727.470688:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5128 00000100:00100000:1.0:1789760727.470698:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705852840289 time 705908738259 state 0x2 ref 0 00000001:00080000:0.0:1789760727.470961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972416 is committed 00000100:00100000:1.0:1789760727.474919:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.475070:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650560 00000100:00100000:0.0:1789760727.475080:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650560 00000100:00100000:0.0:1789760727.475140:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705852840289 time 705908738259 state 0x0 ref 1 00000100:00100000:0.0:1789760727.475155:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705908998675 time 705908738259 state 0x1 ref 1 00000100:00100000:0.0:1789760727.475221:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705908998675 time 705940509850 state 0x0 ref 1 00000100:00100000:0.0:1789760727.475234:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515650560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.475279:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20889600, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:0.0:1789760727.475287:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.475309:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760727.475313:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.475319:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.475332:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.505632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972417 is committed 00002000:00010000:0.0:1789760727.505868:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20889600 -> 20893696 00010000:00010000:0.0:1789760727.505884:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.505897:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.505906:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.505913:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.505918:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.505927:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.505946:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.505955:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.506055:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515650560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30818us (31141us total) trans 4294972417 rc 0/0 00000100:00100000:0.0:1789760727.506065:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5129 00000100:00100000:0.0:1789760727.506075:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705908998675 time 705940509850 state 0x2 ref 0 00000100:00100000:1.0:1789760727.509322:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760727.509603:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650688 00000100:00100000:2.0:1789760727.509614:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650688 00000100:00100000:2.0:1789760727.509665:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705908998675 time 705940509850 state 0x0 ref 1 00000100:00100000:2.0:1789760727.509680:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705940770266 time 705940509850 state 0x1 ref 1 00000100:00100000:2.0:1789760727.509727:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705940770266 time 705975015414 state 0x0 ref 1 00000100:00100000:2.0:1789760727.509747:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515650688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760727.509796:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20893696, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:2.0:1789760727.509806:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.509822:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760727.509826:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.509832:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.509847:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.562137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972418 is committed 00002000:00010000:3.0:1789760727.562288:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20893696 -> 20897792 00010000:00010000:3.0:1789760727.562304:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.562317:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.562327:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.562335:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760727.562340:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.562349:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.562373:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.562380:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dde5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.562534:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515650688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52784us (53217us total) trans 4294972418 rc 0/0 00000100:00100000:3.0:1789760727.562544:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5130 00000100:00100000:3.0:1789760727.562553:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705940770266 time 705975015414 state 0x2 ref 0 00000100:00100000:2.0:1789760727.565218:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.565331:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650816 00000100:00100000:0.0:1789760727.565341:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650816 00000100:00100000:0.0:1789760727.565399:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705940770266 time 705975015414 state 0x0 ref 1 00000100:00100000:0.0:1789760727.565412:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705975275830 time 705975015414 state 0x1 ref 1 00000100:00100000:0.0:1789760727.565441:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705975275830 time 706030730057 state 0x0 ref 1 00000100:00100000:0.0:1789760727.565452:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515650816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.565499:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20897792, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:0.0:1789760727.565506:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.565521:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760727.565525:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.565531:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.565548:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.616284:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20897792 -> 20901888 00010000:00010000:1.0:1789760727.616305:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.616317:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.616325:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.616333:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760727.616337:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.616345:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.616438:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.616446:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ddec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.616570:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515650816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51114us (51358us total) trans 4294972419 rc 0/0 00000100:00100000:1.0:1789760727.616580:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5131 00000100:00100000:1.0:1789760727.616590:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705975275830 time 706030730057 state 0x2 ref 0 00000001:00080000:1.0:1789760727.616698:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972419 is committed 00000100:00100000:1.0:1789760727.621022:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760727.621245:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515650944 00000100:00100000:2.0:1789760727.621257:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515650944 00000100:00100000:2.0:1789760727.621309:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 705975275830 time 706030730057 state 0x0 ref 1 00000100:00100000:2.0:1789760727.621323:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706030990473 time 706030730057 state 0x1 ref 1 00000100:00100000:2.0:1789760727.621458:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706030990473 time 706086745678 state 0x0 ref 1 00000100:00100000:2.0:1789760727.621471:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515650944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760727.621516:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20901888, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:2.0:1789760727.621525:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.621540:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.621544:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.621550:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.621564:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.656245:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20901888 -> 20905984 00010000:00010000:0.0:1789760727.656267:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.656280:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.656290:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.656297:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.656302:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.656312:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.656330:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.656338:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ddf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.656443:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515650944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34969us (35425us total) trans 4294972420 rc 0/0 00000100:00100000:0.0:1789760727.656453:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5132 00000100:00100000:0.0:1789760727.656462:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706030990473 time 706086745678 state 0x2 ref 0 00000100:00100000:0.0:1789760727.661355:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760727.661493:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651072 00000100:00100000:2.0:1789760727.661504:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651072 00000100:00100000:2.0:1789760727.661555:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706030990473 time 706086745678 state 0x0 ref 1 00000100:00100000:2.0:1789760727.661572:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706087006094 time 706086745678 state 0x1 ref 1 00000100:00100000:2.0:1789760727.661676:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706087006094 time 706126963159 state 0x0 ref 1 00000100:00100000:2.0:1789760727.661690:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515651072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760727.661767:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20905984, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:2.0:1789760727.661776:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.661797:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.661802:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.661808:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.661822:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.662613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972420 is committed 00000001:00080000:1.0:1789760727.698267:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972421 is committed 00002000:00010000:0.0:1789760727.698356:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20905984 -> 20910080 00010000:00010000:0.0:1789760727.698372:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.698385:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.698394:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.698401:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.698406:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.698415:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.698433:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.698447:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ddfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.698537:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515651072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36845us (37186us total) trans 4294972421 rc 0/0 00000100:00100000:0.0:1789760727.698547:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5133 00000100:00100000:0.0:1789760727.698556:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706087006094 time 706126963159 state 0x2 ref 0 00000100:00100000:2.0:1789760727.702558:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760727.702809:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651200 00000100:00100000:1.0:1789760727.702817:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651200 00000100:00100000:1.0:1789760727.702866:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706087006094 time 706126963159 state 0x0 ref 1 00000100:00100000:1.0:1789760727.702881:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706127223575 time 706126963159 state 0x1 ref 1 00000100:00100000:1.0:1789760727.702910:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706127223575 time 706168199754 state 0x0 ref 1 00000100:00100000:1.0:1789760727.702920:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760727.702962:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20910080, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:1.0:1789760727.702970:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.702985:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760727.702989:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.702994:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.703112:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.741141:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972422 is committed 00002000:00010000:3.0:1789760727.741299:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20910080 -> 20914176 00010000:00010000:3.0:1789760727.741316:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741330:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741339:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741347:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760727.741353:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741366:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741385:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.741392:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1de01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.741486:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38562us (38933us total) trans 4294972422 rc 0/0 00000100:00100000:3.0:1789760727.741496:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5134 00000100:00100000:3.0:1789760727.741505:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706127223575 time 706168199754 state 0x2 ref 0 00000100:00100000:1.0:1789760727.746412:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760727.746495:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651328 00000100:00100000:2.0:1789760727.746505:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651328 00000100:00100000:2.0:1789760727.746557:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706127223575 time 706168199754 state 0x0 ref 1 00000100:00100000:2.0:1789760727.746571:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706168460170 time 706168199754 state 0x1 ref 1 00000100:00100000:2.0:1789760727.746609:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706168460170 time 706211898725 state 0x0 ref 1 00000100:00100000:2.0:1789760727.746622:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760727.746960:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20914176, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:3.0:1789760727.746971:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.746988:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.746992:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.746998:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.747022:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.785759:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972423 is committed 00002000:00010000:0.0:1789760727.786014:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20914176 -> 20918272 00010000:00010000:0.0:1789760727.786030:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.786042:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.786051:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.786059:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760727.786064:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.786074:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.786108:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.786240:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1de08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.786332:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39708us (39925us total) trans 4294972423 rc 0/0 00000100:00100000:0.0:1789760727.786342:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5135 00000100:00100000:0.0:1789760727.786352:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706168460170 time 706211898725 state 0x2 ref 0 00000100:00100000:1.0:1789760727.790373:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760727.790448:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651456 00000100:00100000:3.0:1789760727.790457:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651456 00000100:00100000:3.0:1789760727.790509:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706168460170 time 706211898725 state 0x0 ref 1 00000100:00100000:3.0:1789760727.790524:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706212159141 time 706211898725 state 0x1 ref 1 00000100:00100000:3.0:1789760727.790552:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706212159141 time 706255841024 state 0x0 ref 1 00000100:00100000:3.0:1789760727.790564:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760727.790619:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20918272, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:3.0:1789760727.790628:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.790644:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.790649:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.790656:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.790669:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.825286:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972424 is committed 00002000:00010000:3.0:1789760727.825461:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20918272 -> 20922368 00010000:00010000:3.0:1789760727.825476:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.825495:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.825504:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.825511:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760727.825516:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.827242:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.827277:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.827286:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.827416:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36848us (37046us total) trans 4294972424 rc 0/0 00000100:00100000:3.0:1789760727.827427:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5136 00000100:00100000:3.0:1789760727.827437:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706212159141 time 706255841024 state 0x2 ref 0 00000100:00100000:2.0:1789760727.832382:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760727.832945:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651584 00000100:00100000:1.0:1789760727.832955:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651584 00000100:00100000:1.0:1789760727.833008:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706212159141 time 706255841024 state 0x0 ref 1 00000100:00100000:1.0:1789760727.833023:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706256101440 time 706255841024 state 0x1 ref 1 00000100:00100000:1.0:1789760727.833107:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706256101440 time 706298395141 state 0x0 ref 1 00000100:00100000:1.0:1789760727.833121:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515651584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760727.833174:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20922368, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xa007 00010000:00010000:1.0:1789760727.833184:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.833200:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760727.833204:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.833211:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.833225:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.887559:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20922368 -> 20926464 00002000:00010000:3.0:1789760727.887569:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760727 -> 1789760728 00002000:00010000:3.0:1789760727.887571:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760727 -> 1789760728 00010000:00010000:3.0:1789760727.887584:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.887598:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.887607:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.887615:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760727.887619:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.887628:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.887647:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.887654:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1de16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.887911:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515651584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54788us (55533us total) trans 4294972425 rc 0/0 00000100:00100000:3.0:1789760727.887921:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5137 00000100:00100000:3.0:1789760727.887931:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706256101440 time 706298395141 state 0x2 ref 0 00000001:00080000:0.0:1789760727.888482:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972425 is committed 00000100:00100000:1.0:1789760727.892593:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.892735:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651712 00000100:00100000:0.0:1789760727.892745:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651712 00000100:00100000:0.0:1789760727.892795:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706256101440 time 706298395141 state 0x0 ref 1 00000100:00100000:0.0:1789760727.892810:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706298655557 time 706298395141 state 0x1 ref 1 00000100:00100000:0.0:1789760727.892842:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706298655557 time 706358131251 state 0x0 ref 1 00000100:00100000:0.0:1789760727.892855:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.892902:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20926464, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:0.0:1789760727.892910:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.892933:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760727.892937:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.892943:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.892958:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.968402:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972426 is committed 00002000:00010000:1.0:1789760727.968511:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20926464 -> 20930560 00010000:00010000:1.0:1789760727.968525:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.968538:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.968548:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.968556:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760727.968561:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.968570:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.968588:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.968596:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1de1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.968700:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515651712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75843us (76112us total) trans 4294972426 rc 0/0 00000100:00100000:1.0:1789760727.968737:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5138 00000100:00100000:1.0:1789760727.968746:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706298655557 time 706358131251 state 0x2 ref 0 00000100:00100000:1.0:1789760727.971803:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760727.972377:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651840 00000100:00100000:0.0:1789760727.972388:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651840 00000100:00100000:0.0:1789760727.972442:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706298655557 time 706358131251 state 0x0 ref 1 00000100:00100000:0.0:1789760727.972457:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706358391667 time 706358131251 state 0x1 ref 1 00000100:00100000:0.0:1789760727.972486:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706358391667 time 706437775426 state 0x0 ref 1 00000100:00100000:0.0:1789760727.972497:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515651840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760727.972543:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20930560, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:0.0:1789760727.972553:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.972568:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760727.972573:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.972579:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.972594:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.003503:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20930560 -> 20934656 00010000:00010000:1.0:1789760728.003525:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003539:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003547:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003559:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.003563:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003573:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003592:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003600:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1de24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.003692:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515651840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31192us (31892us total) trans 4294972427 rc 0/0 00000100:00100000:1.0:1789760728.003703:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5139 00000100:00100000:1.0:1789760728.003742:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706358391667 time 706437775426 state 0x2 ref 0 00000001:00080000:3.0:1789760728.003826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972427 is committed 00000100:00100000:0.0:1789760728.007268:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.007358:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515651968 00000100:00100000:3.0:1789760728.007368:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515651968 00000100:00100000:3.0:1789760728.007431:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706358391667 time 706437775426 state 0x0 ref 1 00000100:00100000:3.0:1789760728.007446:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706438035842 time 706437775426 state 0x1 ref 1 00000100:00100000:3.0:1789760728.007479:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706438035842 time 706472768169 state 0x0 ref 1 00000100:00100000:3.0:1789760728.007491:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515651968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.007538:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20934656, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:3.0:1789760728.007546:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.007562:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760728.007566:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.007573:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.007587:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760728.008046:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760728.062735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972428 is committed 00002000:00010000:3.0:1789760728.063030:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20934656 -> 20938752 00010000:00010000:3.0:1789760728.063046:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.063064:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.063073:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.063079:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760728.063097:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.063107:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.063126:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.063179:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.063258:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515651968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55763us (55994us total) trans 4294972428 rc 0/0 00000100:00100000:3.0:1789760728.063270:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5140 00000100:00100000:3.0:1789760728.063280:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706438035842 time 706472768169 state 0x2 ref 0 00000100:00100000:2.0:1789760728.066395:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.066480:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652096 00000100:00100000:3.0:1789760728.066491:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652096 00000100:00100000:3.0:1789760728.066543:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706438035842 time 706472768169 state 0x0 ref 1 00000100:00100000:3.0:1789760728.066558:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706473028585 time 706472768169 state 0x1 ref 1 00000100:00100000:3.0:1789760728.066632:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706473028585 time 706531920619 state 0x0 ref 1 00000100:00100000:3.0:1789760728.066645:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515652096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.066684:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20938752, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:3.0:1789760728.066690:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.066702:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760728.066734:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.066742:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.066757:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.109006:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972429 is committed 00002000:00010000:1.0:1789760728.109847:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20938752 -> 20942848 00010000:00010000:1.0:1789760728.109864:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.109879:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.109890:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.109898:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.109902:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.109912:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.109933:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.109942:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1de32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.110046:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515652096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43398us (43652us total) trans 4294972429 rc 0/0 00000100:00100000:1.0:1789760728.110057:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5141 00000100:00100000:1.0:1789760728.110067:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706473028585 time 706531920619 state 0x2 ref 0 00000100:00100000:1.0:1789760728.122402:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760728.122499:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652224 00000100:00100000:2.0:1789760728.122508:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652224 00000100:00100000:2.0:1789760728.122560:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706473028585 time 706531920619 state 0x0 ref 1 00000100:00100000:2.0:1789760728.122579:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706532181035 time 706531920619 state 0x1 ref 1 00000100:00100000:2.0:1789760728.122608:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706532181035 time 706587896680 state 0x0 ref 1 00000100:00100000:2.0:1789760728.122619:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515652224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760728.122663:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20942848, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:2.0:1789760728.122671:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.122687:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760728.122692:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.122698:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.122749:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.157992:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972430 is committed 00002000:00010000:2.0:1789760728.158110:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20942848 -> 20946944 00010000:00010000:2.0:1789760728.158846:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.158860:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.158869:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.158881:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760728.158885:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.158893:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.158910:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.158916:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1de39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.159015:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515652224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36393us (36618us total) trans 4294972430 rc 0/0 00000100:00100000:2.0:1789760728.159025:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5142 00000100:00100000:2.0:1789760728.159034:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706532181035 time 706587896680 state 0x2 ref 0 00000100:00100000:0.0:1789760728.164496:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.164747:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652352 00000100:00100000:3.0:1789760728.164758:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652352 00000100:00100000:3.0:1789760728.164823:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706532181035 time 706587896680 state 0x0 ref 1 00000100:00100000:3.0:1789760728.164840:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706588157096 time 706587896680 state 0x1 ref 1 00000100:00100000:3.0:1789760728.164870:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706588157096 time 706630159457 state 0x0 ref 1 00000100:00100000:3.0:1789760728.164884:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515652352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.164933:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20946944, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:3.0:1789760728.164941:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.164955:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760728.164960:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.164966:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.164979:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.211329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972431 is committed 00002000:00010000:0.0:1789760728.212424:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20946944 -> 20951040 00010000:00010000:0.0:1789760728.212440:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.212454:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.212463:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.212472:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760728.212476:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.212487:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.212508:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.212522:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1de40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.212623:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515652352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47735us (48133us total) trans 4294972431 rc 0/0 00000100:00100000:0.0:1789760728.212633:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5143 00000100:00100000:0.0:1789760728.212642:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706588157096 time 706630159457 state 0x2 ref 0 00000100:00100000:1.0:1789760728.216844:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760728.217326:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652480 00000100:00100000:0.0:1789760728.217335:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652480 00000100:00100000:0.0:1789760728.217399:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706588157096 time 706630159457 state 0x0 ref 1 00000100:00100000:0.0:1789760728.217413:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706630419873 time 706630159457 state 0x1 ref 1 00000100:00100000:0.0:1789760728.217446:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706630419873 time 706682734631 state 0x0 ref 1 00000100:00100000:0.0:1789760728.217458:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515652480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760728.217503:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20951040, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:0.0:1789760728.217512:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.217527:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760728.217531:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.217538:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.217558:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.262267:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972432 is committed 00002000:00010000:2.0:1789760728.262413:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20951040 -> 20955136 00010000:00010000:2.0:1789760728.262429:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.262441:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.262455:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.262462:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760728.262468:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.262477:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.262496:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.262504:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1de47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.262594:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515652480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45133us (45755us total) trans 4294972432 rc 0/0 00000100:00100000:2.0:1789760728.262603:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5144 00000100:00100000:2.0:1789760728.262612:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706630419873 time 706682734631 state 0x2 ref 0 00000100:00100000:2.0:1789760728.268111:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760728.268212:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652608 00000100:00100000:1.0:1789760728.268221:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652608 00000100:00100000:1.0:1789760728.268264:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706630419873 time 706682734631 state 0x0 ref 1 00000100:00100000:1.0:1789760728.268285:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706682995047 time 706682734631 state 0x1 ref 1 00000100:00100000:1.0:1789760728.268306:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706682995047 time 706733596148 state 0x0 ref 1 00000100:00100000:1.0:1789760728.268319:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515652608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760728.268433:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20955136, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:1.0:1789760728.268442:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.268458:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760728.268463:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.268469:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.268483:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.300364:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972433 is committed 00002000:00010000:0.0:1789760728.300749:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20955136 -> 20959232 00010000:00010000:0.0:1789760728.300764:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.300775:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.300785:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.300792:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760728.300796:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.300805:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.300825:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.300833:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1de4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.300920:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515652608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32598us (32813us total) trans 4294972433 rc 0/0 00000100:00100000:0.0:1789760728.300930:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5145 00000100:00100000:0.0:1789760728.300940:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706682995047 time 706733596148 state 0x2 ref 0 00000100:00100000:0.0:1789760728.306447:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760728.306875:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652736 00000100:00100000:1.0:1789760728.306884:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652736 00000100:00100000:1.0:1789760728.306939:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706682995047 time 706733596148 state 0x0 ref 1 00000100:00100000:1.0:1789760728.306955:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706733856564 time 706733596148 state 0x1 ref 1 00000100:00100000:1.0:1789760728.306981:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706733856564 time 706772270356 state 0x0 ref 1 00000100:00100000:1.0:1789760728.307009:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515652736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760728.307066:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20959232, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:1.0:1789760728.307076:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.307102:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760728.307106:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.307112:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.307127:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.342691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972434 is committed 00002000:00010000:1.0:1789760728.342816:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20959232 -> 20963328 00010000:00010000:1.0:1789760728.342832:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342845:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342855:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342863:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.342868:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342877:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342898:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.342906:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.343008:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515652736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35997us (36558us total) trans 4294972434 rc 0/0 00000100:00100000:1.0:1789760728.343018:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5146 00000100:00100000:1.0:1789760728.343027:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706733856564 time 706772270356 state 0x2 ref 0 00000100:00100000:1.0:1789760728.349127:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.349237:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652864 00000100:00100000:3.0:1789760728.349246:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652864 00000100:00100000:3.0:1789760728.349300:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706733856564 time 706772270356 state 0x0 ref 1 00000100:00100000:3.0:1789760728.349314:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706772530772 time 706772270356 state 0x1 ref 1 00000100:00100000:3.0:1789760728.349346:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706772530772 time 706814635471 state 0x0 ref 1 00000100:00100000:3.0:1789760728.349357:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515652864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.349402:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20963328, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:3.0:1789760728.349410:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.349434:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760728.349439:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.349446:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.349462:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.382108:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760728.391877:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20963328 -> 20967424 00010000:00010000:3.0:1789760728.391897:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.391911:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.391919:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.391927:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760728.391932:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.391941:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.391965:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.391973:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.392074:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515652864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42712us (42950us total) trans 4294972435 rc 0/0 00000100:00100000:3.0:1789760728.392093:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5147 00000100:00100000:3.0:1789760728.392104:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706772530772 time 706814635471 state 0x2 ref 0 00000001:00080000:1.0:1789760728.392917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972435 is committed 00000100:00100000:3.0:1789760728.396072:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760728.396255:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515652992 00000100:00100000:2.0:1789760728.396267:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515652992 00000100:00100000:2.0:1789760728.396325:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706772530772 time 706814635471 state 0x0 ref 1 00000100:00100000:2.0:1789760728.396344:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706814895887 time 706814635471 state 0x1 ref 1 00000100:00100000:2.0:1789760728.396373:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706814895887 time 706861662173 state 0x0 ref 1 00000100:00100000:2.0:1789760728.396385:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515652992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760728.396434:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20967424, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:2.0:1789760728.396443:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.396460:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760728.396465:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.396471:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.396485:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.428445:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20967424 -> 20971520 00010000:00010000:1.0:1789760728.428466:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.428479:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.428488:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.428495:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.428500:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.428510:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.428529:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.428536:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1de63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.428634:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515652992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32246us (32566us total) trans 4294972436 rc 0/0 00000100:00100000:1.0:1789760728.428644:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5148 00000100:00100000:1.0:1789760728.428655:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706814895887 time 706861662173 state 0x2 ref 0 00000001:00080000:3.0:1789760728.429066:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972436 is committed 00000100:00100000:0.0:1789760728.440335:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.440489:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653120 00000100:00100000:3.0:1789760728.440499:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653120 00000100:00100000:3.0:1789760728.440561:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706814895887 time 706861662173 state 0x0 ref 1 00000100:00100000:3.0:1789760728.440576:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706861922589 time 706861662173 state 0x1 ref 1 00000100:00100000:3.0:1789760728.440649:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706861922589 time 706905937529 state 0x0 ref 1 00000100:00100000:3.0:1789760728.440663:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.440738:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20971520, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa007 00010000:00010000:3.0:1789760728.440747:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.440769:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760728.440773:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.440779:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.440793:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.501174:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972437 is committed 00002000:00010000:0.0:1789760728.501460:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20971520 -> 20975616 00002000:00010000:0.0:1789760728.501465:0:13072:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:0.0:1789760728.501479:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.501493:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.501502:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.501511:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760728.501515:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.501525:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.501549:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.501557:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1de6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.501648:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60982us (62841us total) trans 4294972437 rc 0/0 00000100:00100000:0.0:1789760728.501658:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5149 00000100:00100000:0.0:1789760728.501668:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706861922589 time 706905937529 state 0x2 ref 0 00000100:00100000:0.0:1789760728.524822:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760728.525398:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653248 00000100:00100000:2.0:1789760728.525410:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653248 00000100:00100000:2.0:1789760728.525465:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706861922589 time 706905937529 state 0x0 ref 1 00000100:00100000:2.0:1789760728.525482:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706906197945 time 706905937529 state 0x1 ref 1 00000100:00100000:2.0:1789760728.525557:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706906197945 time 706990845447 state 0x0 ref 1 00000100:00100000:2.0:1789760728.525571:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515653248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760728.525618:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20975616, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:2.0:1789760728.525627:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.525747:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760728.525753:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.525759:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.525774:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.560480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972438 is committed 00002000:00010000:1.0:1789760728.560594:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20975616 -> 20979712 00010000:00010000:1.0:1789760728.560611:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.560625:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.560634:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.560642:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.560647:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.560657:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.560678:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.560685:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1de71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.560822:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515653248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35249us (36004us total) trans 4294972438 rc 0/0 00000100:00100000:1.0:1789760728.560834:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5150 00000100:00100000:1.0:1789760728.560843:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706906197945 time 706990845447 state 0x2 ref 0 00000100:00100000:3.0:1789760728.565261:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760728.565561:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653376 00000100:00100000:0.0:1789760728.565572:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653376 00000100:00100000:0.0:1789760728.565624:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706906197945 time 706990845447 state 0x0 ref 1 00000100:00100000:0.0:1789760728.565640:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706991105863 time 706990845447 state 0x1 ref 1 00000100:00100000:0.0:1789760728.565678:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706991105863 time 707030967202 state 0x0 ref 1 00000100:00100000:0.0:1789760728.565700:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515653376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760728.565768:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20979712, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:0.0:1789760728.565777:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.565798:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.565802:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.565809:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.565822:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.573960:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544409600:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760728.574041:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760728.574139:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544409728:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760728.574181:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760728.574807:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409600 00000100:00100000:0.0:1789760728.574815:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409600 00000100:00100000:0.0:1789760728.574871:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409728 00000100:00100000:0.0:1789760728.574875:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409728 00000100:00100000:0.0:1789760728.574903:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789760728.574913:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544409600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760728.574937:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3837788160 avail 3814719488 00000100:00100000:0.0:1789760728.574987:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544409600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (946us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.574994:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789760728.575017:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1789760728.575024:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544409728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760728.575047:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760728.575095:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544409728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 72us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.575102:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00080000:1.0:1789760728.575167:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760728.575171:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760728.575344:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544409600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760728.575645:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544409728:0@lo:13:osp-pre-1-0.0 00002000:00010000:0.0:1789760728.604199:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20979712 -> 20983808 00010000:00010000:0.0:1789760728.604221:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.604236:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.604246:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.604255:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760728.604260:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.604270:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.604289:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.604298:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1de78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.604404:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515653376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38701us (39147us total) trans 4294972439 rc 0/0 00000100:00100000:0.0:1789760728.604415:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5151 00000100:00100000:0.0:1789760728.604423:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706991105863 time 707030967202 state 0x2 ref 0 00000001:00080000:3.0:1789760728.605119:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972439 is committed 00000100:00100000:3.0:1789760728.614265:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760728.614530:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653504 00000100:00100000:1.0:1789760728.614540:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653504 00000100:00100000:1.0:1789760728.614591:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 706991105863 time 707030967202 state 0x0 ref 1 00000100:00100000:1.0:1789760728.614606:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707031227618 time 707030967202 state 0x1 ref 1 00000100:00100000:1.0:1789760728.614919:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707031227618 time 707080206217 state 0x0 ref 1 00000100:00100000:1.0:1789760728.614932:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760728.614980:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20983808, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:1.0:1789760728.614989:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.615136:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.615209:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.615217:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.615233:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.653227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972440 is committed 00002000:00010000:2.0:1789760728.653332:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20983808 -> 20987904 00010000:00010000:2.0:1789760728.653348:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.653362:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.653371:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.653378:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760728.653384:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.653393:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.653411:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.653420:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.653548:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38613us (39287us total) trans 4294972440 rc 0/0 00000100:00100000:2.0:1789760728.653559:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5152 00000100:00100000:2.0:1789760728.653568:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707031227618 time 707080206217 state 0x2 ref 0 00000100:00100000:2.0:1789760728.660569:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.660650:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653632 00000100:00100000:3.0:1789760728.660660:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653632 00000100:00100000:3.0:1789760728.660818:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707031227618 time 707080206217 state 0x0 ref 1 00000100:00100000:3.0:1789760728.660835:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707080466633 time 707080206217 state 0x1 ref 1 00000100:00100000:3.0:1789760728.660867:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707080466633 time 707126155913 state 0x0 ref 1 00000100:00100000:3.0:1789760728.660880:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515653632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.660932:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20987904, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:3.0:1789760728.660942:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.660959:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.660964:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.660970:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.660985:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760728.687226:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20987904 -> 20992000 00010000:00010000:3.0:1789760728.687248:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687261:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687270:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687277:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760728.687282:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687292:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687313:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687321:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1de86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.687410:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515653632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26527us (26845us total) trans 4294972441 rc 0/0 00000100:00100000:3.0:1789760728.687420:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5153 00000100:00100000:3.0:1789760728.687429:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707080466633 time 707126155913 state 0x2 ref 0 00000001:00080000:0.0:1789760728.688633:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972441 is committed 00000100:00100000:2.0:1789760728.693097:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760728.693375:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653760 00000100:00100000:1.0:1789760728.693386:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653760 00000100:00100000:1.0:1789760728.693441:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707080466633 time 707126155913 state 0x0 ref 1 00000100:00100000:1.0:1789760728.693455:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707126416329 time 707126155913 state 0x1 ref 1 00000100:00100000:1.0:1789760728.693484:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707126416329 time 707158773495 state 0x0 ref 1 00000100:00100000:1.0:1789760728.693497:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760728.693545:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20992000, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:1.0:1789760728.693555:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.693571:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.693576:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.693582:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.693595:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.749622:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972442 is committed 00002000:00010000:2.0:1789760728.749877:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20992000 -> 20996096 00010000:00010000:2.0:1789760728.749898:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.749911:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.749920:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.749929:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760728.749934:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.749942:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.749963:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.749971:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1de8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.750092:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56583us (57005us total) trans 4294972442 rc 0/0 00000100:00100000:2.0:1789760728.750102:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5154 00000100:00100000:2.0:1789760728.750112:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707126416329 time 707158773495 state 0x2 ref 0 00000100:00100000:1.0:1789760728.763892:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760728.764179:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515653888 00000100:00100000:0.0:1789760728.764190:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515653888 00000100:00100000:0.0:1789760728.764246:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707126416329 time 707158773495 state 0x0 ref 1 00000100:00100000:0.0:1789760728.764262:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707159033911 time 707158773495 state 0x1 ref 1 00000100:00100000:0.0:1789760728.764293:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707159033911 time 707229582753 state 0x0 ref 1 00000100:00100000:0.0:1789760728.764306:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760728.764354:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 20996096, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:0.0:1789760728.764363:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.764379:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.764384:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.764390:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.764403:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.811036:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972443 is committed 00002000:00010000:0.0:1789760728.811196:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 20996096 -> 21000192 00010000:00010000:0.0:1789760728.811214:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.811234:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.811242:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.811249:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760728.811253:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.811261:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.811463:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.811472:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1de94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.811564:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515653888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47255us (47678us total) trans 4294972443 rc 0/0 00000100:00100000:0.0:1789760728.811575:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5155 00000100:00100000:0.0:1789760728.811584:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707159033911 time 707229582753 state 0x2 ref 0 00000100:00100000:0.0:1789760728.816841:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760728.816919:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654016 00000100:00100000:1.0:1789760728.816929:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654016 00000100:00100000:1.0:1789760728.816989:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707159033911 time 707229582753 state 0x0 ref 1 00000100:00100000:1.0:1789760728.817130:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707229843169 time 707229582753 state 0x1 ref 1 00000100:00100000:1.0:1789760728.817160:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707229843169 time 707282449733 state 0x0 ref 1 00000100:00100000:1.0:1789760728.817173:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515654016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760728.817217:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21000192, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xa807 00010000:00010000:1.0:1789760728.817226:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.817242:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.817247:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.817253:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.817267:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.853882:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21000192 -> 21004288 00002000:00010000:1.0:1789760728.853893:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760728 -> 1789760729 00002000:00010000:1.0:1789760728.853895:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760728 -> 1789760729 00010000:00010000:1.0:1789760728.853908:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.853921:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.853930:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.853937:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.853943:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.853952:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.853972:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.853980:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1de9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.854218:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515654016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37042us (37381us total) trans 4294972444 rc 0/0 00000100:00100000:1.0:1789760728.854229:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5156 00000100:00100000:1.0:1789760728.854238:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707229843169 time 707282449733 state 0x2 ref 0 00000001:00080000:0.0:1789760728.854840:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972444 is committed 00000100:00100000:3.0:1789760728.858845:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760728.859231:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654144 00000100:00100000:0.0:1789760728.859241:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654144 00000100:00100000:0.0:1789760728.859293:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707229843169 time 707282449733 state 0x0 ref 1 00000100:00100000:0.0:1789760728.859318:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707282710149 time 707282449733 state 0x1 ref 1 00000100:00100000:0.0:1789760728.859344:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707282710149 time 707324633553 state 0x0 ref 1 00000100:00100000:0.0:1789760728.859354:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760728.859399:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21004288, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:0.0:1789760728.859409:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.859424:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.859429:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.859435:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.859449:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.892474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972445 is committed 00002000:00010000:0.0:1789760728.892562:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21004288 -> 21008384 00010000:00010000:0.0:1789760728.892577:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.892589:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.892597:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.892605:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760728.892609:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.892617:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.892635:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.892643:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dea2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.892792:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33434us (33951us total) trans 4294972445 rc 0/0 00000100:00100000:0.0:1789760728.892803:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5157 00000100:00100000:0.0:1789760728.892812:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707282710149 time 707324633553 state 0x2 ref 0 00000100:00100000:3.0:1789760728.902524:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760728.902628:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654272 00000100:00100000:2.0:1789760728.902638:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654272 00000100:00100000:2.0:1789760728.902696:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707282710149 time 707324633553 state 0x0 ref 1 00000100:00100000:2.0:1789760728.902801:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707324893969 time 707324633553 state 0x1 ref 1 00000100:00100000:2.0:1789760728.902870:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707324893969 time 707368158942 state 0x0 ref 1 00000100:00100000:2.0:1789760728.902887:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515654272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760728.902939:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21008384, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760728.902948:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.902964:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760728.902968:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.902974:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.903004:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.936849:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21008384 -> 21012480 00010000:00010000:1.0:1789760728.936870:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.936883:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.936892:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.936900:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760728.936905:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.936914:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.936932:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.936946:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1dea9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.937048:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515654272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34158us (34527us total) trans 4294972446 rc 0/0 00000100:00100000:1.0:1789760728.937058:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5158 00000100:00100000:1.0:1789760728.937067:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707324893969 time 707368158942 state 0x2 ref 0 00000001:00080000:0.0:1789760728.938347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972446 is committed 00000100:00100000:2.0:1789760728.942149:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760728.942234:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654400 00000100:00100000:3.0:1789760728.942244:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654400 00000100:00100000:3.0:1789760728.942295:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707324893969 time 707368158942 state 0x0 ref 1 00000100:00100000:3.0:1789760728.942309:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707368419358 time 707368158942 state 0x1 ref 1 00000100:00100000:3.0:1789760728.942338:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707368419358 time 707407627130 state 0x0 ref 1 00000100:00100000:3.0:1789760728.942349:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760728.942397:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21012480, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760728.942405:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.942421:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.942426:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.942432:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.942445:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.998203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972447 is committed 00002000:00010000:2.0:1789760728.998305:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21012480 -> 21016576 00010000:00010000:2.0:1789760728.998320:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.998333:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.998341:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.998349:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760728.998355:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.998364:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.998384:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.998392:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1deb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.998480:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56129us (56336us total) trans 4294972447 rc 0/0 00000100:00100000:2.0:1789760728.998491:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5159 00000100:00100000:2.0:1789760728.998501:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707368419358 time 707407627130 state 0x2 ref 0 00000100:00100000:1.0:1789760729.003018:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.003102:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654528 00000100:00100000:3.0:1789760729.003111:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654528 00000100:00100000:3.0:1789760729.003159:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707368419358 time 707407627130 state 0x0 ref 1 00000100:00100000:3.0:1789760729.003173:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707407887546 time 707407627130 state 0x1 ref 1 00000100:00100000:3.0:1789760729.003289:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707407887546 time 707468577551 state 0x0 ref 1 00000100:00100000:3.0:1789760729.003302:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.003350:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21016576, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.003358:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.003374:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760729.003380:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.003386:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.003405:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.029846:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972448 is committed 00002000:00010000:3.0:1789760729.029982:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21016576 -> 21020672 00010000:00010000:3.0:1789760729.030060:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.030082:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.030093:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.030101:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.030107:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.030116:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.030136:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.030144:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.030280:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26975us (27265us total) trans 4294972448 rc 0/0 00000100:00100000:3.0:1789760729.030291:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5160 00000100:00100000:3.0:1789760729.030301:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707407887546 time 707468577551 state 0x2 ref 0 00000100:00100000:1.0:1789760729.034186:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760729.034280:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654656 00000100:00100000:0.0:1789760729.034290:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654656 00000100:00100000:0.0:1789760729.034345:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707407887546 time 707468577551 state 0x0 ref 1 00000100:00100000:0.0:1789760729.034364:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707468837967 time 707468577551 state 0x1 ref 1 00000100:00100000:0.0:1789760729.034424:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707468837967 time 707499712539 state 0x0 ref 1 00000100:00100000:0.0:1789760729.034437:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760729.034483:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21020672, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:0.0:1789760729.034493:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.034510:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760729.034514:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.034521:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.034535:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.073612:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972449 is committed 00002000:00010000:1.0:1789760729.073828:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21020672 -> 21024768 00010000:00010000:1.0:1789760729.073844:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.073856:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.073866:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.073873:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.073878:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.073893:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.073912:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.073920:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1debe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.074123:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39683us (39941us total) trans 4294972449 rc 0/0 00000100:00100000:1.0:1789760729.074133:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5161 00000100:00100000:1.0:1789760729.074143:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707468837967 time 707499712539 state 0x2 ref 0 00000100:00100000:2.0:1789760729.078108:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760729.078197:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654784 00000100:00100000:1.0:1789760729.078206:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654784 00000100:00100000:1.0:1789760729.078259:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707468837967 time 707499712539 state 0x0 ref 1 00000100:00100000:1.0:1789760729.078277:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707499972955 time 707499712539 state 0x1 ref 1 00000100:00100000:1.0:1789760729.078341:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707499972955 time 707543629729 state 0x0 ref 1 00000100:00100000:1.0:1789760729.078353:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760729.078403:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21024768, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:1.0:1789760729.078412:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.078429:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760729.078434:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.078440:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.078453:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.107542:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972450 is committed 00002000:00010000:2.0:1789760729.107650:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21024768 -> 21028864 00010000:00010000:2.0:1789760729.107668:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.107681:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.107691:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.107699:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.107704:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.107750:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.107768:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.107775:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1dec5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.107854:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29497us (29749us total) trans 4294972450 rc 0/0 00000100:00100000:2.0:1789760729.107864:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5162 00000100:00100000:2.0:1789760729.107875:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707499972955 time 707543629729 state 0x2 ref 0 00000100:00100000:2.0:1789760729.111574:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760729.111661:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515654912 00000100:00100000:1.0:1789760729.111671:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515654912 00000100:00100000:1.0:1789760729.111765:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707499972955 time 707543629729 state 0x0 ref 1 00000100:00100000:1.0:1789760729.111781:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707543890145 time 707543629729 state 0x1 ref 1 00000100:00100000:1.0:1789760729.111801:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707543890145 time 707577090964 state 0x0 ref 1 00000100:00100000:1.0:1789760729.111814:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760729.111861:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21028864, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:1.0:1789760729.111870:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.111885:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760729.111890:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.111896:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.111911:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.137119:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972451 is committed 00002000:00010000:2.0:1789760729.137262:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21028864 -> 21032960 00010000:00010000:2.0:1789760729.137277:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.137290:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.137305:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.137312:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.137317:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.137326:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.137344:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.137351:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1decc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.137442:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515654912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25626us (25872us total) trans 4294972451 rc 0/0 00000100:00100000:2.0:1789760729.137451:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5163 00000100:00100000:2.0:1789760729.137459:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707543890145 time 707577090964 state 0x2 ref 0 00000100:00100000:1.0:1789760729.140098:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.140211:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655040 00000100:00100000:2.0:1789760729.140221:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655040 00000100:00100000:2.0:1789760729.140270:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707543890145 time 707577090964 state 0x0 ref 1 00000100:00100000:2.0:1789760729.140284:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707577351380 time 707577090964 state 0x1 ref 1 00000100:00100000:2.0:1789760729.140328:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707577351380 time 707605616021 state 0x0 ref 1 00000100:00100000:2.0:1789760729.140340:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515655040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.140386:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21032960, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.140403:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.140418:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760729.140422:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.140428:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.140442:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760729.165865:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21032960 -> 21037056 00010000:00010000:2.0:1789760729.165886:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.165899:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.165909:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.165917:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.165923:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.165931:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.165950:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.165958:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ded3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.166059:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515655040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25717us (25964us total) trans 4294972452 rc 0/0 00000100:00100000:2.0:1789760729.166071:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5164 00000100:00100000:2.0:1789760729.166089:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707577351380 time 707605616021 state 0x2 ref 0 00000001:00080000:0.0:1789760729.168967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972452 is committed 00000100:00100000:1.0:1789760729.170280:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.170356:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655168 00000100:00100000:2.0:1789760729.170366:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655168 00000100:00100000:2.0:1789760729.170418:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707577351380 time 707605616021 state 0x0 ref 1 00000100:00100000:2.0:1789760729.170433:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707605876437 time 707605616021 state 0x1 ref 1 00000100:00100000:2.0:1789760729.170464:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707605876437 time 707635753253 state 0x0 ref 1 00000100:00100000:2.0:1789760729.170476:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.170525:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21037056, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.170535:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.170552:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760729.170557:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.170563:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.170577:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.199520:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972453 is committed 00002000:00010000:2.0:1789760729.199636:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21037056 -> 21041152 00010000:00010000:2.0:1789760729.199652:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.199665:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.199674:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.199682:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.199687:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.199696:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.199810:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.199819:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1deda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.199949:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29469us (29670us total) trans 4294972453 rc 0/0 00000100:00100000:2.0:1789760729.199961:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5165 00000100:00100000:2.0:1789760729.199969:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707605876437 time 707635753253 state 0x2 ref 0 00000100:00100000:1.0:1789760729.203440:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.203538:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655296 00000100:00100000:3.0:1789760729.203548:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655296 00000100:00100000:3.0:1789760729.203599:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707605876437 time 707635753253 state 0x0 ref 1 00000100:00100000:3.0:1789760729.203614:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707636013669 time 707635753253 state 0x1 ref 1 00000100:00100000:3.0:1789760729.203682:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707636013669 time 707668969329 state 0x0 ref 1 00000100:00100000:3.0:1789760729.203695:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.203819:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21041152, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.203828:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.203843:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760729.203847:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.203854:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.203867:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.225408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972454 is committed 00002000:00010000:3.0:1789760729.225413:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21041152 -> 21045248 00010000:00010000:3.0:1789760729.225430:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.225443:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.225451:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.225460:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.225465:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.225473:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.225492:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.225500:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1dee1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.225584:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21888us (22150us total) trans 4294972454 rc 0/0 00000100:00100000:3.0:1789760729.225595:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5166 00000100:00100000:3.0:1789760729.225603:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707636013669 time 707668969329 state 0x2 ref 0 00000100:00100000:1.0:1789760729.231853:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760729.231951:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655424 00000100:00100000:0.0:1789760729.231961:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655424 00000100:00100000:0.0:1789760729.232022:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707636013669 time 707668969329 state 0x0 ref 1 00000100:00100000:0.0:1789760729.232037:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707669229745 time 707668969329 state 0x1 ref 1 00000100:00100000:0.0:1789760729.232183:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707669229745 time 707697467401 state 0x0 ref 1 00000100:00100000:0.0:1789760729.232201:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760729.232283:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21045248, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:0.0:1789760729.232295:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.232320:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760729.232330:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.232467:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.232493:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.271283:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972455 is committed 00002000:00010000:2.0:1789760729.271561:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21045248 -> 21049344 00010000:00010000:2.0:1789760729.271576:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271590:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271599:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271605:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.271610:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271622:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271644:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.271652:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dee8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.271832:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39630us (39986us total) trans 4294972455 rc 0/0 00000100:00100000:2.0:1789760729.271842:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5167 00000100:00100000:2.0:1789760729.271852:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707669229745 time 707697467401 state 0x2 ref 0 00000100:00100000:2.0:1789760729.275191:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.275284:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655552 00000100:00100000:3.0:1789760729.275295:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655552 00000100:00100000:3.0:1789760729.275349:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707669229745 time 707697467401 state 0x0 ref 1 00000100:00100000:3.0:1789760729.275365:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707697727817 time 707697467401 state 0x1 ref 1 00000100:00100000:3.0:1789760729.275398:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707697727817 time 707740686972 state 0x0 ref 1 00000100:00100000:3.0:1789760729.275410:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.275457:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21049344, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.275465:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.275479:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760729.275484:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.275489:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.275501:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.299338:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972456 is committed 00002000:00010000:3.0:1789760729.299480:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21049344 -> 21053440 00010000:00010000:3.0:1789760729.299497:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.299511:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.299520:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.299527:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.299532:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.299541:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.299560:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.299568:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1deef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.299935:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515655552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24521us (24748us total) trans 4294972456 rc 0/0 00000100:00100000:3.0:1789760729.299947:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5168 00000100:00100000:3.0:1789760729.299957:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707697727817 time 707740686972 state 0x2 ref 0 00000100:00100000:1.0:1789760729.304819:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.304966:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655680 00000100:00100000:2.0:1789760729.304976:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655680 00000100:00100000:2.0:1789760729.305045:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707697727817 time 707740686972 state 0x0 ref 1 00000100:00100000:2.0:1789760729.305061:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707740947388 time 707740686972 state 0x1 ref 1 00000100:00100000:2.0:1789760729.305098:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707740947388 time 707770386619 state 0x0 ref 1 00000100:00100000:2.0:1789760729.305112:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515655680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.305162:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21053440, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.305171:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.305187:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760729.305191:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.305198:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.305211:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.338021:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972457 is committed 00002000:00010000:3.0:1789760729.338251:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21053440 -> 21057536 00010000:00010000:3.0:1789760729.338267:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.338280:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.338289:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.338297:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.338302:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.338310:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.338327:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.338335:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1def6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.338456:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515655680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33342us (33644us total) trans 4294972457 rc 0/0 00000100:00100000:3.0:1789760729.338468:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5169 00000100:00100000:3.0:1789760729.338478:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707740947388 time 707770386619 state 0x2 ref 0 00000100:00100000:1.0:1789760729.341429:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.341552:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655808 00000100:00100000:2.0:1789760729.341562:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655808 00000100:00100000:2.0:1789760729.341614:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707740947388 time 707770386619 state 0x0 ref 1 00000100:00100000:2.0:1789760729.341631:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707770647035 time 707770386619 state 0x1 ref 1 00000100:00100000:2.0:1789760729.341745:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707770647035 time 707807033457 state 0x0 ref 1 00000100:00100000:2.0:1789760729.341758:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515655808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.341802:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21057536, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.341812:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.341908:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760729.341916:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.341924:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.341940:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.371592:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972458 is committed 00002000:00010000:3.0:1789760729.371876:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21057536 -> 21061632 00010000:00010000:3.0:1789760729.371894:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.371907:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.371917:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.371925:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.371930:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.371940:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.371962:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.371970:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1defd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.372136:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515655808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30375us (30711us total) trans 4294972458 rc 0/0 00000100:00100000:3.0:1789760729.372146:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5170 00000100:00100000:3.0:1789760729.372156:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707770647035 time 707807033457 state 0x2 ref 0 00000100:00100000:2.0:1789760729.375819:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760729.376031:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515655936 00000100:00100000:0.0:1789760729.376042:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515655936 00000100:00100000:0.0:1789760729.376108:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707770647035 time 707807033457 state 0x0 ref 1 00000100:00100000:0.0:1789760729.376125:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707807293873 time 707807033457 state 0x1 ref 1 00000100:00100000:0.0:1789760729.376185:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707807293873 time 707841474136 state 0x0 ref 1 00000100:00100000:0.0:1789760729.376199:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515655936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760729.376248:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21061632, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:0.0:1789760729.376257:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.376273:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760729.376277:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.376282:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.376295:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.401335:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972459 is committed 00002000:00010000:0.0:1789760729.401542:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21061632 -> 21065728 00010000:00010000:0.0:1789760729.401556:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.401569:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.401579:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.401586:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760729.401592:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.401601:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.401626:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.401634:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1df04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.401946:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515655936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25744us (26133us total) trans 4294972459 rc 0/0 00000100:00100000:0.0:1789760729.401958:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5171 00000100:00100000:0.0:1789760729.401968:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707807293873 time 707841474136 state 0x2 ref 0 00000100:00100000:1.0:1789760729.406558:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.406653:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656064 00000100:00100000:3.0:1789760729.406662:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656064 00000100:00100000:3.0:1789760729.406758:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707807293873 time 707841474136 state 0x0 ref 1 00000100:00100000:3.0:1789760729.406774:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707841734552 time 707841474136 state 0x1 ref 1 00000100:00100000:3.0:1789760729.406806:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707841734552 time 707872095324 state 0x0 ref 1 00000100:00100000:3.0:1789760729.406820:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.406865:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21065728, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.406874:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.406890:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760729.406894:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.406900:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.406913:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.411304:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760729.430760:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21065728 -> 21069824 00010000:00010000:2.0:1789760729.430779:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.430793:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.430802:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.430810:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.430815:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.430825:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.430844:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.430852:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1df0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.430942:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515656064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24119us (24387us total) trans 4294972460 rc 0/0 00000100:00100000:2.0:1789760729.430953:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5172 00000100:00100000:2.0:1789760729.430962:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707841734552 time 707872095324 state 0x2 ref 0 00000001:00080000:1.0:1789760729.432507:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972460 is committed 00000100:00100000:0.0:1789760729.435612:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.438848:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656192 00000100:00100000:2.0:1789760729.438864:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656192 00000100:00100000:2.0:1789760729.438933:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707841734552 time 707872095324 state 0x0 ref 1 00000100:00100000:2.0:1789760729.438952:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707872355740 time 707872095324 state 0x1 ref 1 00000100:00100000:2.0:1789760729.439206:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707872355740 time 707904493775 state 0x0 ref 1 00000100:00100000:2.0:1789760729.439221:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.439274:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21069824, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.439284:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.439300:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760729.439305:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.439311:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.439328:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.469037:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972461 is committed 00002000:00010000:1.0:1789760729.469243:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21069824 -> 21073920 00010000:00010000:1.0:1789760729.469258:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.469271:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.469280:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.469288:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.469293:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.469303:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.469323:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.469332:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1df12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.469424:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30200us (33815us total) trans 4294972461 rc 0/0 00000100:00100000:1.0:1789760729.469436:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5173 00000100:00100000:1.0:1789760729.469445:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707872355740 time 707904493775 state 0x2 ref 0 00000100:00100000:0.0:1789760729.472582:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.472683:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656320 00000100:00100000:3.0:1789760729.472693:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656320 00000100:00100000:3.0:1789760729.472869:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707872355740 time 707904493775 state 0x0 ref 1 00000100:00100000:3.0:1789760729.472883:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707904754191 time 707904493775 state 0x1 ref 1 00000100:00100000:3.0:1789760729.472944:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707904754191 time 707938232811 state 0x0 ref 1 00000100:00100000:3.0:1789760729.472956:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.473067:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21073920, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.473084:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.473101:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760729.473106:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.473113:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.473128:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.498207:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21073920 -> 21078016 00010000:00010000:1.0:1789760729.498228:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.498242:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.498251:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.498260:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.498266:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.498276:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.498296:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.498305:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1df19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.498398:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515656320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25439us (25820us total) trans 4294972462 rc 0/0 00000100:00100000:1.0:1789760729.498409:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5174 00000100:00100000:1.0:1789760729.498418:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707904754191 time 707938232811 state 0x2 ref 0 00000001:00080000:2.0:1789760729.498489:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972462 is committed 00000100:00100000:0.0:1789760729.502343:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.502437:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656448 00000100:00100000:2.0:1789760729.502448:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656448 00000100:00100000:2.0:1789760729.502500:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707904754191 time 707938232811 state 0x0 ref 1 00000100:00100000:2.0:1789760729.502515:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707938493227 time 707938232811 state 0x1 ref 1 00000100:00100000:2.0:1789760729.502581:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707938493227 time 707967869873 state 0x0 ref 1 00000100:00100000:2.0:1789760729.502594:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.502647:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21078016, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.502657:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.502672:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760729.502676:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.502683:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.502697:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.528810:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972463 is committed 00002000:00010000:1.0:1789760729.528932:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21078016 -> 21082112 00010000:00010000:1.0:1789760729.528948:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.528960:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.528969:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.528983:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.528988:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.528997:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.529016:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.529024:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1df20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.529155:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26559us (26818us total) trans 4294972463 rc 0/0 00000100:00100000:1.0:1789760729.529166:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5175 00000100:00100000:1.0:1789760729.529176:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707938493227 time 707967869873 state 0x2 ref 0 00000100:00100000:0.0:1789760729.533676:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.533822:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656576 00000100:00100000:3.0:1789760729.533832:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656576 00000100:00100000:3.0:1789760729.533884:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707938493227 time 707967869873 state 0x0 ref 1 00000100:00100000:3.0:1789760729.533900:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707968130289 time 707967869873 state 0x1 ref 1 00000100:00100000:3.0:1789760729.533928:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707968130289 time 707999216872 state 0x0 ref 1 00000100:00100000:3.0:1789760729.533941:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.533991:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21082112, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.534002:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.534017:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760729.534022:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.534028:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.534042:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.552583:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972464 is committed 00002000:00010000:1.0:1789760729.552668:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21082112 -> 21086208 00010000:00010000:1.0:1789760729.552683:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.552696:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.552778:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.552785:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.552790:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.552799:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.552820:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.552828:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1df27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.553081:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 19128us (19401us total) trans 4294972464 rc 0/0 00000100:00100000:2.0:1789760729.553093:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5176 00000100:00100000:2.0:1789760729.553102:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707968130289 time 707999216872 state 0x2 ref 0 00000100:00100000:0.0:1789760729.557337:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.557475:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656704 00000100:00100000:2.0:1789760729.557487:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656704 00000100:00100000:2.0:1789760729.557550:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707968130289 time 707999216872 state 0x0 ref 1 00000100:00100000:2.0:1789760729.557569:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707999477288 time 707999216872 state 0x1 ref 1 00000100:00100000:2.0:1789760729.557638:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707999477288 time 708022927137 state 0x0 ref 1 00000100:00100000:2.0:1789760729.557650:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515656704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.557694:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21086208, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.557702:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.557838:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760729.557842:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.557849:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.557863:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.588224:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972465 is committed 00002000:00010000:0.0:1789760729.588309:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21086208 -> 21090304 00010000:00010000:0.0:1789760729.588323:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.588335:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.588343:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.588350:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760729.588355:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.588363:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.588381:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.588389:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1df2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.588511:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515656704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30859us (31179us total) trans 4294972465 rc 0/0 00000100:00100000:0.0:1789760729.588522:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5177 00000100:00100000:0.0:1789760729.588531:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707999477288 time 708022927137 state 0x2 ref 0 00000100:00100000:0.0:1789760729.592036:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.592158:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656832 00000100:00100000:3.0:1789760729.592168:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656832 00000100:00100000:3.0:1789760729.592216:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 707999477288 time 708022927137 state 0x0 ref 1 00000100:00100000:3.0:1789760729.592232:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708023187553 time 708022927137 state 0x1 ref 1 00000100:00100000:3.0:1789760729.592262:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708023187553 time 708057551378 state 0x0 ref 1 00000100:00100000:3.0:1789760729.592273:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515656832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.592319:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21090304, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.592328:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.592343:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760729.592347:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.592354:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.592367:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.623596:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21090304 -> 21094400 00010000:00010000:3.0:1789760729.623616:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.623629:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.623638:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.623646:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.623651:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.623661:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.623681:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.623690:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.623954:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515656832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31677us (31923us total) trans 4294972466 rc 0/0 00000100:00100000:3.0:1789760729.623965:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5178 00000100:00100000:3.0:1789760729.623976:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708023187553 time 708057551378 state 0x2 ref 0 00000100:00100000:0.0:1789760729.628055:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:2.0:1789760729.628243:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972466 is committed 00000100:00100000:1.0:1789760729.628327:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515656960 00000100:00100000:1.0:1789760729.628337:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515656960 00000100:00100000:1.0:1789760729.628390:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708023187553 time 708057551378 state 0x0 ref 1 00000100:00100000:1.0:1789760729.628405:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708057811794 time 708057551378 state 0x1 ref 1 00000100:00100000:1.0:1789760729.628471:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708057811794 time 708093759899 state 0x0 ref 1 00000100:00100000:1.0:1789760729.628485:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515656960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760729.628533:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21094400, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:1.0:1789760729.628542:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.628559:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.628564:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.628570:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.628583:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.652625:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972467 is committed 00002000:00010000:1.0:1789760729.652826:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21094400 -> 21098496 00010000:00010000:1.0:1789760729.652842:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.652855:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.652865:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.652873:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.652878:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.652888:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.652908:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.652916:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.653066:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515656960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24575us (25067us total) trans 4294972467 rc 0/0 00000100:00100000:1.0:1789760729.653086:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5179 00000100:00100000:1.0:1789760729.653096:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708057811794 time 708093759899 state 0x2 ref 0 00000100:00100000:1.0:1789760729.656422:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.656500:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657088 00000100:00100000:3.0:1789760729.656509:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657088 00000100:00100000:3.0:1789760729.656559:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708057811794 time 708093759899 state 0x0 ref 1 00000100:00100000:3.0:1789760729.656574:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708094020315 time 708093759899 state 0x1 ref 1 00000100:00100000:3.0:1789760729.656616:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708094020315 time 708121902460 state 0x0 ref 1 00000100:00100000:3.0:1789760729.656632:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.656677:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21098496, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.656687:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.656703:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760729.656831:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.656838:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.656853:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.682944:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972468 is committed 00002000:00010000:3.0:1789760729.683063:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21098496 -> 21102592 00010000:00010000:3.0:1789760729.683088:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.683101:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.683111:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.683118:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.683123:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.683133:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.683153:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.683160:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.683249:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26617us (26831us total) trans 4294972468 rc 0/0 00000100:00100000:3.0:1789760729.683263:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5180 00000100:00100000:3.0:1789760729.683274:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708094020315 time 708121902460 state 0x2 ref 0 00000100:00100000:0.0:1789760729.687201:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760729.687302:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657216 00000100:00100000:1.0:1789760729.687313:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657216 00000100:00100000:1.0:1789760729.687364:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708094020315 time 708121902460 state 0x0 ref 1 00000100:00100000:1.0:1789760729.687379:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708122162876 time 708121902460 state 0x1 ref 1 00000100:00100000:1.0:1789760729.687441:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708122162876 time 708152729640 state 0x0 ref 1 00000100:00100000:1.0:1789760729.687455:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760729.687500:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21102592, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:1.0:1789760729.687509:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.687530:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.687535:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.687541:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.687554:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.717389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972469 is committed 00002000:00010000:2.0:1789760729.717489:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21102592 -> 21106688 00010000:00010000:2.0:1789760729.717504:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.717517:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.717526:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.717533:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.717537:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.717545:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.717563:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.717569:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1df4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.717663:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30206us (30466us total) trans 4294972469 rc 0/0 00000100:00100000:2.0:1789760729.717674:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5181 00000100:00100000:2.0:1789760729.717685:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708122162876 time 708152729640 state 0x2 ref 0 00000100:00100000:0.0:1789760729.721087:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760729.721228:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657344 00000100:00100000:2.0:1789760729.721238:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657344 00000100:00100000:2.0:1789760729.721293:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708122162876 time 708152729640 state 0x0 ref 1 00000100:00100000:2.0:1789760729.721308:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708152990056 time 708152729640 state 0x1 ref 1 00000100:00100000:2.0:1789760729.721329:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708152990056 time 708186618845 state 0x0 ref 1 00000100:00100000:2.0:1789760729.721340:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760729.721385:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21106688, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:2.0:1789760729.721394:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.721408:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760729.721413:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.721419:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.721433:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.767212:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972470 is committed 00002000:00010000:1.0:1789760729.767330:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21106688 -> 21110784 00010000:00010000:1.0:1789760729.767346:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767360:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767370:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767378:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.767383:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767393:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767490:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.767500:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1df51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.767595:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46252us (46519us total) trans 4294972470 rc 0/0 00000100:00100000:1.0:1789760729.767606:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5182 00000100:00100000:1.0:1789760729.768009:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708152990056 time 708186618845 state 0x2 ref 0 00000100:00100000:1.0:1789760729.772146:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760729.772333:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657472 00000100:00100000:0.0:1789760729.772343:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657472 00000100:00100000:0.0:1789760729.772396:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708152990056 time 708186618845 state 0x0 ref 1 00000100:00100000:0.0:1789760729.772426:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708186879261 time 708186618845 state 0x1 ref 1 00000100:00100000:0.0:1789760729.772457:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708186879261 time 708237746099 state 0x0 ref 1 00000100:00100000:0.0:1789760729.772470:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760729.772514:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21110784, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:0.0:1789760729.772524:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.772539:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.772544:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.772551:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.772563:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.807515:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972471 is committed 00002000:00010000:1.0:1789760729.807616:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21110784 -> 21114880 00010000:00010000:1.0:1789760729.807633:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.807647:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.807655:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.807663:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760729.807668:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.807677:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.807702:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.807739:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1df58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.807824:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35351us (35681us total) trans 4294972471 rc 0/0 00000100:00100000:1.0:1789760729.807834:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5183 00000100:00100000:1.0:1789760729.807844:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708186879261 time 708237746099 state 0x2 ref 0 00000100:00100000:0.0:1789760729.811915:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.812010:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657600 00000100:00100000:3.0:1789760729.812019:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657600 00000100:00100000:3.0:1789760729.812082:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708186879261 time 708237746099 state 0x0 ref 1 00000100:00100000:3.0:1789760729.812097:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708238006515 time 708237746099 state 0x1 ref 1 00000100:00100000:3.0:1789760729.812124:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708238006515 time 708277413005 state 0x0 ref 1 00000100:00100000:3.0:1789760729.812136:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.812183:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21114880, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa807 00010000:00010000:3.0:1789760729.812192:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.812208:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760729.812212:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.812218:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.812231:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.855795:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972472 is committed 00002000:00010000:2.0:1789760729.855897:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21114880 -> 21118976 00002000:00010000:2.0:1789760729.855902:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760729 -> 1789760730 00002000:00010000:2.0:1789760729.855905:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760729 -> 1789760730 00010000:00010000:2.0:1789760729.855918:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855932:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855941:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855949:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760729.855954:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855964:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855985:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.856066:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.856232:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44092us (44321us total) trans 4294972472 rc 0/0 00000100:00100000:2.0:1789760729.856243:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5184 00000100:00100000:2.0:1789760729.856253:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708238006515 time 708277413005 state 0x2 ref 0 00000100:00100000:3.0:1789760729.862081:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760729.862273:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657728 00000100:00100000:1.0:1789760729.862284:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657728 00000100:00100000:1.0:1789760729.862336:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708238006515 time 708277413005 state 0x0 ref 1 00000100:00100000:1.0:1789760729.862351:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708277673421 time 708277413005 state 0x1 ref 1 00000100:00100000:1.0:1789760729.862385:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708277673421 time 708327674054 state 0x0 ref 1 00000100:00100000:1.0:1789760729.862397:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760729.862444:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21118976, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:1.0:1789760729.862453:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.862474:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.862479:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.862485:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.862499:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.979907:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972473 is committed 00002000:00010000:3.0:1789760729.980139:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21118976 -> 21123072 00010000:00010000:3.0:1789760729.980154:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.980167:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.980175:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.980181:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760729.980185:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.980194:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.980212:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.980220:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.980313:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 117912us (118246us total) trans 4294972473 rc 0/0 00000100:00100000:3.0:1789760729.980323:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5185 00000100:00100000:3.0:1789760729.980332:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708277673421 time 708327674054 state 0x2 ref 0 00000100:00100000:1.0:1789760729.985975:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760729.986121:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657856 00000100:00100000:3.0:1789760729.986132:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657856 00000100:00100000:3.0:1789760729.986182:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708277673421 time 708327674054 state 0x0 ref 1 00000100:00100000:3.0:1789760729.986198:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708327934470 time 708327674054 state 0x1 ref 1 00000100:00100000:3.0:1789760729.986229:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708327934470 time 708451517840 state 0x0 ref 1 00000100:00100000:3.0:1789760729.986241:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760729.986288:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21123072, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:3.0:1789760729.986298:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986314:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760729.986319:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.986326:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.986341:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.033944:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972474 is committed 00002000:00010000:2.0:1789760730.034190:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21123072 -> 21127168 00010000:00010000:2.0:1789760730.034205:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.034223:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.034233:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.034240:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760730.034245:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.034253:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.034272:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.034279:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1df6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.034440:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515657856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48197us (48470us total) trans 4294972474 rc 0/0 00000100:00100000:2.0:1789760730.034449:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5186 00000100:00100000:2.0:1789760730.034458:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708327934470 time 708451517840 state 0x2 ref 0 00000100:00100000:2.0:1789760730.042680:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.042918:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515657984 00000100:00100000:1.0:1789760730.042929:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515657984 00000100:00100000:1.0:1789760730.042997:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708327934470 time 708451517840 state 0x0 ref 1 00000100:00080000:2.0:1789760730.043004:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760730.043012:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708451778256 time 708451517840 state 0x1 ref 1 00000100:00100000:1.0:1789760730.043044:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708451778256 time 708508332596 state 0x0 ref 1 00000100:00100000:1.0:1789760730.043057:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.043191:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21127168, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:1.0:1789760730.043201:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.043216:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760730.043220:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.043227:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.043241:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.089231:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972475 is committed 00002000:00010000:1.0:1789760730.089335:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21127168 -> 21131264 00010000:00010000:1.0:1789760730.089351:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.089365:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.089375:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.089383:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.089389:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.089398:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.089418:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.089425:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.089522:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515657984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46464us (46847us total) trans 4294972475 rc 0/0 00000100:00100000:1.0:1789760730.089533:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5187 00000100:00100000:1.0:1789760730.089543:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708451778256 time 708508332596 state 0x2 ref 0 00000100:00100000:0.0:1789760730.092964:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.093059:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658112 00000100:00100000:1.0:1789760730.093079:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658112 00000100:00100000:1.0:1789760730.093132:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708451778256 time 708508332596 state 0x0 ref 1 00000100:00100000:1.0:1789760730.093149:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708508593012 time 708508332596 state 0x1 ref 1 00000100:00100000:1.0:1789760730.093182:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708508593012 time 708558470597 state 0x0 ref 1 00000100:00100000:1.0:1789760730.093195:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515658112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.093245:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21131264, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:1.0:1789760730.093255:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.093270:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760730.093276:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.093283:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.093298:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.145361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972476 is committed 00002000:00010000:2.0:1789760730.145461:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21131264 -> 21135360 00010000:00010000:2.0:1789760730.145476:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.145545:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.145555:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.145563:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760730.145568:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.145577:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.145597:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.145605:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1df7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.145697:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515658112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52500us (52738us total) trans 4294972476 rc 0/0 00000100:00100000:2.0:1789760730.149878:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5188 00000100:00100000:2.0:1789760730.149893:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708508593012 time 708558470597 state 0x2 ref 0 00000100:00100000:0.0:1789760730.157118:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.157215:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658240 00000100:00100000:1.0:1789760730.157225:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658240 00000100:00100000:1.0:1789760730.157275:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708508593012 time 708558470597 state 0x0 ref 1 00000100:00100000:1.0:1789760730.157290:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708558731013 time 708558470597 state 0x1 ref 1 00000100:00100000:1.0:1789760730.157349:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708558731013 time 708622637294 state 0x0 ref 1 00000100:00100000:1.0:1789760730.157362:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515658240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.157411:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21135360, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:1.0:1789760730.157420:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.157434:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760730.157438:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.157445:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.157460:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760730.207392:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21135360 -> 21139456 00010000:00010000:2.0:1789760730.207411:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.207424:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.207434:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.207442:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760730.207447:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.207457:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.207474:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.207481:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1df82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.207578:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515658240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50212us (50465us total) trans 4294972477 rc 0/0 00000100:00100000:2.0:1789760730.207590:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5189 00000100:00100000:2.0:1789760730.207599:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708558731013 time 708622637294 state 0x2 ref 0 00000001:00080000:3.0:1789760730.208209:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972477 is committed 00000100:00100000:0.0:1789760730.211262:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.211347:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658368 00000100:00100000:2.0:1789760730.211356:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658368 00000100:00100000:2.0:1789760730.211411:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708558731013 time 708622637294 state 0x0 ref 1 00000100:00100000:2.0:1789760730.211426:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708622897710 time 708622637294 state 0x1 ref 1 00000100:00100000:2.0:1789760730.211457:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708622897710 time 708676747020 state 0x0 ref 1 00000100:00100000:2.0:1789760730.211468:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515658368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.211514:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21139456, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.211523:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.211540:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760730.211545:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.211551:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.211575:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.239470:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972478 is committed 00002000:00010000:2.0:1789760730.239671:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21139456 -> 21143552 00010000:00010000:2.0:1789760730.239687:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.239701:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.239757:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.239764:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760730.239769:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.239778:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.239796:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.239804:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.239883:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515658368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28411us (28627us total) trans 4294972478 rc 0/0 00000100:00100000:2.0:1789760730.239898:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5190 00000100:00100000:2.0:1789760730.239907:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708622897710 time 708676747020 state 0x2 ref 0 00000100:00100000:0.0:1789760730.244373:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760730.244455:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658496 00000100:00100000:3.0:1789760730.244465:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658496 00000100:00100000:3.0:1789760730.244516:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708622897710 time 708676747020 state 0x0 ref 1 00000100:00100000:3.0:1789760730.244529:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708677007436 time 708676747020 state 0x1 ref 1 00000100:00100000:3.0:1789760730.244552:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708677007436 time 708709841672 state 0x0 ref 1 00000100:00100000:3.0:1789760730.244562:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515658496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760730.244607:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21143552, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:3.0:1789760730.244617:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.244632:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760730.244637:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.244643:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.244656:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.281318:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21143552 -> 21147648 00010000:00010000:0.0:1789760730.281338:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.281363:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.281372:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.281380:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760730.281384:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.281393:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.281412:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.281419:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1df90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.281510:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515658496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36945us (37152us total) trans 4294972479 rc 0/0 00000100:00100000:0.0:1789760730.281521:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5191 00000100:00100000:0.0:1789760730.281531:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708677007436 time 708709841672 state 0x2 ref 0 00000001:00080000:2.0:1789760730.282544:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972479 is committed 00000100:00100000:1.0:1789760730.286498:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.286579:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658624 00000100:00100000:2.0:1789760730.286588:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658624 00000100:00100000:2.0:1789760730.286653:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708677007436 time 708709841672 state 0x0 ref 1 00000100:00100000:2.0:1789760730.286668:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708710102088 time 708709841672 state 0x1 ref 1 00000100:00100000:2.0:1789760730.286697:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708710102088 time 708751986780 state 0x0 ref 1 00000100:00100000:2.0:1789760730.286844:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515658624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.286894:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21147648, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.286904:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.286920:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760730.286925:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.286930:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.286944:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.350750:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972480 is committed 00002000:00010000:2.0:1789760730.350966:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21147648 -> 21151744 00010000:00010000:2.0:1789760730.350982:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.351005:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.351013:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.351019:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760730.351024:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.351032:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.351052:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.351059:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1df97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.351155:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515658624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64444us (64662us total) trans 4294972480 rc 0/0 00000100:00100000:2.0:1789760730.351165:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5192 00000100:00100000:2.0:1789760730.351174:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708710102088 time 708751986780 state 0x2 ref 0 00000100:00100000:1.0:1789760730.354858:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.354969:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658752 00000100:00100000:2.0:1789760730.354978:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658752 00000100:00100000:2.0:1789760730.355154:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708710102088 time 708751986780 state 0x0 ref 1 00000100:00100000:2.0:1789760730.355170:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708752247196 time 708751986780 state 0x1 ref 1 00000100:00100000:2.0:1789760730.355202:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708752247196 time 708820490631 state 0x0 ref 1 00000100:00100000:2.0:1789760730.355214:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515658752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.355262:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21151744, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.355271:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.355287:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760730.355291:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.355298:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.355312:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.418696:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972481 is committed 00002000:00010000:0.0:1789760730.418922:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21151744 -> 21155840 00010000:00010000:0.0:1789760730.418936:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.418949:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.418959:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.418966:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760730.418971:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.418981:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.419010:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.419018:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1df9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.420462:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+6:2365:x1876699515658752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63906us (64268us total) trans 4294972481 rc 0/0 00000100:00100000:0.0:1789760730.420480:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5193 00000100:00100000:0.0:1789760730.420492:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708752247196 time 708820490631 state 0x2 ref 0 00000100:00100000:1.0:1789760730.422804:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.422968:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515658880 00000100:00100000:2.0:1789760730.422977:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515658880 00000100:00100000:2.0:1789760730.423098:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708752247196 time 708820490631 state 0x0 ref 1 00000100:00100000:2.0:1789760730.423112:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708820751047 time 708820490631 state 0x1 ref 1 00000100:00100000:2.0:1789760730.423257:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708820751047 time 708888544340 state 0x0 ref 1 00000100:00100000:2.0:1789760730.423269:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515658880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.423314:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21155840, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.423323:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.423429:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760730.423435:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.423442:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.423456:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.429888:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760730.465700:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972482 is committed 00002000:00010000:1.0:1789760730.465879:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21155840 -> 21159936 00010000:00010000:1.0:1789760730.465894:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.465907:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.465916:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.465924:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.465928:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.465937:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.465955:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.465962:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1dfa5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.468559:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515658880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45280us (45760us total) trans 4294972482 rc 0/0 00000100:00100000:0.0:1789760730.468573:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5194 00000100:00100000:0.0:1789760730.468582:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708820751047 time 708888544340 state 0x2 ref 0 00000100:00100000:1.0:1789760730.473677:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.473890:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659008 00000100:00100000:2.0:1789760730.473901:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659008 00000100:00100000:2.0:1789760730.473954:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708820751047 time 708888544340 state 0x0 ref 1 00000100:00100000:2.0:1789760730.473969:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708888804756 time 708888544340 state 0x1 ref 1 00000100:00100000:2.0:1789760730.474042:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708888804756 time 708939330071 state 0x0 ref 1 00000100:00100000:2.0:1789760730.474055:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515659008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.474264:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21159936, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.474274:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.474287:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760730.474291:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.474297:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.474310:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.523264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972483 is committed 00002000:00010000:1.0:1789760730.523375:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21159936 -> 21164032 00010000:00010000:1.0:1789760730.523391:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523404:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523412:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523419:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.523424:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523433:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523452:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.523459:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1dfac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.523555:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515659008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49496us (49881us total) trans 4294972483 rc 0/0 00000100:00100000:1.0:1789760730.523565:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5195 00000100:00100000:1.0:1789760730.523575:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708888804756 time 708939330071 state 0x2 ref 0 00000100:00100000:0.0:1789760730.527634:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760730.528086:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659136 00000100:00100000:3.0:1789760730.528096:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659136 00000100:00100000:2.0:1789760730.528309:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708888804756 time 708939330071 state 0x0 ref 1 00000100:00100000:2.0:1789760730.528325:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708939590487 time 708939330071 state 0x1 ref 1 00000100:00100000:2.0:1789760730.528359:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708939590487 time 708993648338 state 0x0 ref 1 00000100:00100000:2.0:1789760730.528371:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.528418:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21164032, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.528427:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.528441:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760730.528445:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.528451:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.528468:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.561675:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972484 is committed 00002000:00010000:2.0:1789760730.562470:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21164032 -> 21168128 00010000:00010000:2.0:1789760730.562489:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.562503:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.562512:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.562520:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760730.562525:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.562534:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.562555:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.562564:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1dfb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.562747:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34330us (35076us total) trans 4294972484 rc 0/0 00000100:00100000:2.0:1789760730.562758:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5196 00000100:00100000:2.0:1789760730.562780:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708939590487 time 708993648338 state 0x2 ref 0 00000100:00100000:2.0:1789760730.568360:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760730.568389:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659264 00000100:00100000:0.0:1789760730.568400:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659264 00000100:00100000:0.0:1789760730.568449:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708939590487 time 708993648338 state 0x0 ref 1 00000100:00100000:0.0:1789760730.568464:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708993908754 time 708993648338 state 0x1 ref 1 00000100:00100000:0.0:1789760730.568495:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708993908754 time 709033784305 state 0x0 ref 1 00000100:00100000:0.0:1789760730.568506:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760730.568551:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21168128, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:0.0:1789760730.568560:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.568575:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.568580:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.568586:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.568605:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.600273:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21168128 -> 21172224 00010000:00010000:0.0:1789760730.600292:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.600304:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.600312:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.600318:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760730.600323:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.600387:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.600405:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.600412:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1dfba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.600500:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515659264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31992us (32145us total) trans 4294972485 rc 0/0 00000100:00100000:0.0:1789760730.600510:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5197 00000100:00100000:0.0:1789760730.600518:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708993908754 time 709033784305 state 0x2 ref 0 00000001:00080000:2.0:1789760730.600929:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972485 is committed 00000100:00100000:0.0:1789760730.605631:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760730.605765:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659392 00000100:00100000:3.0:1789760730.605775:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659392 00000100:00100000:3.0:1789760730.606040:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 708993908754 time 709033784305 state 0x0 ref 1 00000100:00100000:3.0:1789760730.606059:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709034044721 time 709033784305 state 0x1 ref 1 00000100:00100000:3.0:1789760730.606305:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709034044721 time 709071592663 state 0x0 ref 1 00000100:00100000:3.0:1789760730.606318:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515659392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760730.606368:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21172224, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:3.0:1789760730.606377:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.606394:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.606398:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.606404:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.606418:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.649284:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21172224 -> 21176320 00010000:00010000:3.0:1789760730.649305:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.649318:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.649328:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.649339:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760730.649344:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.649352:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.649371:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.649377:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.649445:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972486 is committed 00000100:00100000:3.0:1789760730.649468:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515659392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43147us (43841us total) trans 4294972486 rc 0/0 00000100:00100000:3.0:1789760730.649480:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5198 00000100:00100000:3.0:1789760730.649490:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709034044721 time 709071592663 state 0x2 ref 0 00000100:00100000:3.0:1789760730.654891:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760730.658256:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659520 00000100:00100000:0.0:1789760730.658273:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659520 00000100:00100000:1.0:1789760730.661282:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709034044721 time 709071592663 state 0x0 ref 1 00000100:00100000:1.0:1789760730.661300:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709071853079 time 709071592663 state 0x1 ref 1 00000100:00100000:1.0:1789760730.661391:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709071853079 time 709126678139 state 0x0 ref 1 00000100:00100000:1.0:1789760730.661406:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.661455:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21176320, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:1.0:1789760730.661463:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.661479:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.661484:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.661490:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.661505:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.687226:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972487 is committed 00002000:00010000:1.0:1789760730.687415:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21176320 -> 21180416 00010000:00010000:1.0:1789760730.687442:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.687453:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.687463:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.687471:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.687475:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.687483:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.687501:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.687508:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.687599:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26192us (32711us total) trans 4294972487 rc 0/0 00000100:00100000:1.0:1789760730.687609:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5199 00000100:00100000:1.0:1789760730.687618:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709071853079 time 709126678139 state 0x2 ref 0 00000100:00100000:2.0:1789760730.690832:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760730.690979:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659648 00000100:00100000:3.0:1789760730.691038:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659648 00000100:00100000:3.0:1789760730.691097:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709071853079 time 709126678139 state 0x0 ref 1 00000100:00100000:3.0:1789760730.691113:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709126938555 time 709126678139 state 0x1 ref 1 00000100:00100000:3.0:1789760730.691174:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709126938555 time 709156462906 state 0x0 ref 1 00000100:00100000:3.0:1789760730.691188:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515659648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760730.691233:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21180416, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:3.0:1789760730.691241:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.691256:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.691260:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.691267:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.691280:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.713500:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972488 is committed 00002000:00010000:3.0:1789760730.713667:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21180416 -> 21184512 00010000:00010000:3.0:1789760730.713682:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.713694:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.713703:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.713782:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760730.713787:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.713798:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.713818:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.713826:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dfcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.713948:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515659648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22757us (23121us total) trans 4294972488 rc 0/0 00000100:00100000:3.0:1789760730.713959:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5200 00000100:00100000:3.0:1789760730.713973:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709126938555 time 709156462906 state 0x2 ref 0 00000100:00100000:0.0:1789760730.717505:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.717682:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659776 00000100:00100000:1.0:1789760730.717694:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659776 00000100:00100000:1.0:1789760730.717779:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709126938555 time 709156462906 state 0x0 ref 1 00000100:00100000:1.0:1789760730.717954:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709156723322 time 709156462906 state 0x1 ref 1 00000100:00100000:1.0:1789760730.717984:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709156723322 time 709183273277 state 0x0 ref 1 00000100:00100000:1.0:1789760730.718000:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.718047:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21184512, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:1.0:1789760730.718056:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.718182:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.718188:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.718194:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.718209:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.744845:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972489 is committed 00002000:00010000:3.0:1789760730.745001:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21184512 -> 21188608 00010000:00010000:3.0:1789760730.745017:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.745030:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.745040:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.745047:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760730.745053:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.745070:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.745101:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.745109:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1dfd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.745228:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515659776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27226us (27728us total) trans 4294972489 rc 0/0 00000100:00100000:3.0:1789760730.745238:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5201 00000100:00100000:3.0:1789760730.745248:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709156723322 time 709183273277 state 0x2 ref 0 00000100:00100000:0.0:1789760730.749318:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.749541:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515659904 00000100:00100000:2.0:1789760730.749552:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515659904 00000100:00100000:2.0:1789760730.749608:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709156723322 time 709183273277 state 0x0 ref 1 00000100:00100000:2.0:1789760730.749625:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709183533693 time 709183273277 state 0x1 ref 1 00000100:00100000:2.0:1789760730.749659:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709183533693 time 709214947795 state 0x0 ref 1 00000100:00100000:2.0:1789760730.749672:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515659904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.749887:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21188608, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.749897:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.749913:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.749918:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.749925:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.749939:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.789274:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972490 is committed 00002000:00010000:1.0:1789760730.789398:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21188608 -> 21192704 00010000:00010000:1.0:1789760730.789415:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.789429:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.789438:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.789446:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.789450:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.789460:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.789478:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.789485:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1dfdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.789654:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515659904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39934us (40296us total) trans 4294972490 rc 0/0 00000100:00100000:1.0:1789760730.789664:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5202 00000100:00100000:1.0:1789760730.789673:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709183533693 time 709214947795 state 0x2 ref 0 00000100:00100000:1.0:1789760730.794612:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760730.794697:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660032 00000100:00100000:0.0:1789760730.794740:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660032 00000100:00100000:0.0:1789760730.794800:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709183533693 time 709214947795 state 0x0 ref 1 00000100:00100000:0.0:1789760730.794816:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709215208211 time 709214947795 state 0x1 ref 1 00000100:00100000:0.0:1789760730.794840:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709215208211 time 709260129912 state 0x0 ref 1 00000100:00100000:0.0:1789760730.794853:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515660032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760730.794901:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21192704, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:0.0:1789760730.794910:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.794926:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.794932:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.794938:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.794951:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.819605:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972491 is committed 00002000:00010000:0.0:1789760730.819883:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21192704 -> 21196800 00010000:00010000:0.0:1789760730.819898:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.819912:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.819923:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.819929:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760730.819935:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.819946:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.819964:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.819978:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1dfe4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.820167:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515660032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25311us (25559us total) trans 4294972491 rc 0/0 00000100:00100000:0.0:1789760730.820178:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5203 00000100:00100000:0.0:1789760730.820187:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709215208211 time 709260129912 state 0x2 ref 0 00000100:00100000:3.0:1789760730.823684:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.824150:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660160 00000100:00100000:2.0:1789760730.824161:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660160 00000100:00100000:2.0:1789760730.824210:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709215208211 time 709260129912 state 0x0 ref 1 00000100:00100000:2.0:1789760730.824224:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709260390328 time 709260129912 state 0x1 ref 1 00000100:00100000:2.0:1789760730.824258:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709260390328 time 709289546989 state 0x0 ref 1 00000100:00100000:2.0:1789760730.824270:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515660160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.824318:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21196800, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa807 00010000:00010000:2.0:1789760730.824328:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.824343:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.824354:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.824361:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.824374:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.848921:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972492 is committed 00002000:00010000:0.0:1789760730.849111:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21196800 -> 21200896 00002000:00010000:0.0:1789760730.849116:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760730 -> 1789760731 00002000:00010000:0.0:1789760730.849120:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760730 -> 1789760731 00010000:00010000:0.0:1789760730.849134:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.849147:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.849157:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.849165:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760730.849169:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.849179:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.849199:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.849207:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1dfeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.849336:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515660160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25062us (25655us total) trans 4294972492 rc 0/0 00000100:00100000:0.0:1789760730.849347:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5204 00000100:00100000:0.0:1789760730.849357:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709260390328 time 709289546989 state 0x2 ref 0 00000100:00100000:3.0:1789760730.852684:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.852885:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660288 00000100:00100000:1.0:1789760730.852896:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660288 00000100:00100000:1.0:1789760730.852946:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709260390328 time 709289546989 state 0x0 ref 1 00000100:00100000:1.0:1789760730.852962:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709289807405 time 709289546989 state 0x1 ref 1 00000100:00100000:2.0:1789760730.853081:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709289807405 time 709318367224 state 0x0 ref 1 00000100:00100000:2.0:1789760730.853097:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515660288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.853145:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21200896, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760730.853154:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.853170:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.853174:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.853181:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.853194:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.880945:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21200896 -> 21204992 00010000:00010000:1.0:1789760730.880965:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.880995:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.881004:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.881011:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.881016:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.881024:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.881041:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.881049:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1dff2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.881145:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515660288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28047us (28466us total) trans 4294972493 rc 0/0 00000100:00100000:1.0:1789760730.881158:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5205 00000100:00100000:1.0:1789760730.881168:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709289807405 time 709318367224 state 0x2 ref 0 00000001:00080000:3.0:1789760730.881771:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972493 is committed 00000100:00100000:1.0:1789760730.884840:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760730.884957:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660416 00000100:00100000:3.0:1789760730.884967:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660416 00000100:00100000:3.0:1789760730.885024:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709289807405 time 709318367224 state 0x0 ref 1 00000100:00100000:3.0:1789760730.885038:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709318627640 time 709318367224 state 0x1 ref 1 00000100:00100000:3.0:1789760730.885059:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709318627640 time 709350348773 state 0x0 ref 1 00000100:00100000:3.0:1789760730.885082:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515660416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760730.885130:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21204992, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760730.885138:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.885153:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.885158:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.885164:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.885178:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.917086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972494 is committed 00002000:00010000:1.0:1789760730.917177:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21204992 -> 21209088 00010000:00010000:1.0:1789760730.917192:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.917205:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.917214:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.917220:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.917225:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.917233:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.917250:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.917258:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1dff9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.917378:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515660416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32294us (32542us total) trans 4294972494 rc 0/0 00000100:00100000:1.0:1789760730.917389:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5206 00000100:00100000:1.0:1789760730.917398:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709318627640 time 709350348773 state 0x2 ref 0 00000100:00100000:3.0:1789760730.920618:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.920738:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660544 00000100:00100000:1.0:1789760730.920747:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660544 00000100:00100000:1.0:1789760730.920797:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709318627640 time 709350348773 state 0x0 ref 1 00000100:00100000:1.0:1789760730.920810:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709350609189 time 709350348773 state 0x1 ref 1 00000100:00100000:1.0:1789760730.920830:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709350609189 time 709386119784 state 0x0 ref 1 00000100:00100000:1.0:1789760730.920840:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515660544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.920884:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21209088, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760730.920892:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.920908:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.920912:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.920919:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.920933:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.953675:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21209088 -> 21213184 00010000:00010000:0.0:1789760730.953695:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.953728:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.953737:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.953745:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760730.953750:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.953765:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.953784:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.953792:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e000 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.953879:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515660544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33035us (33265us total) trans 4294972495 rc 0/0 00000100:00100000:0.0:1789760730.953889:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5207 00000100:00100000:0.0:1789760730.953900:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709350609189 time 709386119784 state 0x2 ref 0 00000001:00080000:3.0:1789760730.953972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972495 is committed 00000100:00100000:3.0:1789760730.956961:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760730.957280:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660672 00000100:00100000:1.0:1789760730.957290:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660672 00000100:00100000:1.0:1789760730.957350:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709350609189 time 709386119784 state 0x0 ref 1 00000100:00100000:1.0:1789760730.957366:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709386380200 time 709386119784 state 0x1 ref 1 00000100:00100000:1.0:1789760730.957403:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709386380200 time 709422692718 state 0x0 ref 1 00000100:00100000:1.0:1789760730.957415:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515660672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760730.957460:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21213184, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760730.957468:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.957484:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.957488:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.957494:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.957507:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.986379:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21213184 -> 21217280 00010000:00010000:1.0:1789760730.986400:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.986413:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.986423:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.986431:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760730.986437:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.986447:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.986466:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.986475:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e007 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.986575:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515660672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29156us (29617us total) trans 4294972496 rc 0/0 00000100:00100000:1.0:1789760730.986586:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5208 00000100:00100000:1.0:1789760730.986596:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709386380200 time 709422692718 state 0x2 ref 0 00000001:00080000:3.0:1789760730.986665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972496 is committed 00000100:00100000:1.0:1789760730.993123:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760730.993200:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660800 00000100:00100000:2.0:1789760730.993210:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660800 00000100:00100000:2.0:1789760730.993266:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709386380200 time 709422692718 state 0x0 ref 1 00000100:00100000:2.0:1789760730.993281:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709422953134 time 709422692718 state 0x1 ref 1 00000100:00100000:2.0:1789760730.993316:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709422953134 time 709458604392 state 0x0 ref 1 00000100:00100000:2.0:1789760730.993328:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515660800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760730.993374:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21217280, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760730.993383:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.993398:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.993402:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.993408:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.993421:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.023601:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972497 is committed 00002000:00010000:0.0:1789760731.023687:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21217280 -> 21221376 00010000:00010000:0.0:1789760731.023704:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.023738:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.023747:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.023755:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760731.023761:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.023771:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.023790:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.023797:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e00e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.023932:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515660800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30599us (30811us total) trans 4294972497 rc 0/0 00000100:00100000:0.0:1789760731.023943:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5209 00000100:00100000:0.0:1789760731.023954:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709422953134 time 709458604392 state 0x2 ref 0 00000100:00100000:0.0:1789760731.030360:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.030551:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515660928 00000100:00100000:3.0:1789760731.030561:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515660928 00000100:00100000:3.0:1789760731.030614:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709422953134 time 709458604392 state 0x0 ref 1 00000100:00100000:3.0:1789760731.030629:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709458864808 time 709458604392 state 0x1 ref 1 00000100:00100000:3.0:1789760731.030657:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709458864808 time 709495946615 state 0x0 ref 1 00000100:00100000:3.0:1789760731.030673:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515660928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760731.032530:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21221376, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760731.032547:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.032565:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760731.032571:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.032577:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.032590:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.068030:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972498 is committed 00002000:00010000:2.0:1789760731.068305:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21221376 -> 21225472 00010000:00010000:2.0:1789760731.068320:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.068334:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.068344:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.068352:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760731.068357:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.068367:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.068388:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.068396:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e015 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.068488:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515660928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37812us (38132us total) trans 4294972498 rc 0/0 00000100:00100000:2.0:1789760731.068572:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5210 00000100:00100000:2.0:1789760731.068584:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709458864808 time 709495946615 state 0x2 ref 0 00000100:00100000:3.0:1789760731.071489:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760731.071641:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661056 00000100:00100000:2.0:1789760731.071654:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661056 00000100:00100000:2.0:1789760731.071745:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709458864808 time 709495946615 state 0x0 ref 1 00000100:00100000:2.0:1789760731.071762:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709496207031 time 709495946615 state 0x1 ref 1 00000100:00100000:2.0:1789760731.071790:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709496207031 time 709537079505 state 0x0 ref 1 00000100:00100000:2.0:1789760731.071803:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515661056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760731.071853:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21225472, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760731.071862:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.071878:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760731.071883:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.071890:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.071904:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.105480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972499 is committed 00002000:00010000:2.0:1789760731.105646:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21225472 -> 21229568 00010000:00010000:2.0:1789760731.105661:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.105674:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.105689:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.105697:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760731.105793:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.105804:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.105824:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.105834:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e01c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.105924:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+6:2365:x1876699515661056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34118us (34439us total) trans 4294972499 rc 0/0 00000100:00100000:2.0:1789760731.105936:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5211 00000100:00100000:2.0:1789760731.105946:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709496207031 time 709537079505 state 0x2 ref 0 00000100:00100000:0.0:1789760731.111351:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.112278:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661184 00000100:00100000:1.0:1789760731.112290:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661184 00000100:00100000:1.0:1789760731.112347:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709496207031 time 709537079505 state 0x0 ref 1 00000100:00100000:1.0:1789760731.112363:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709537339921 time 709537079505 state 0x1 ref 1 00000100:00100000:1.0:1789760731.112394:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709537339921 time 709577683315 state 0x0 ref 1 00000100:00100000:1.0:1789760731.112407:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515661184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.112459:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21229568, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.112467:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.112483:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760731.112488:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.112621:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.112640:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.158944:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21229568 -> 21233664 00010000:00010000:3.0:1789760731.158963:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.158986:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.158997:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.159004:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.159009:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.159018:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.159036:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.159044:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e023 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.159154:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515661184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46745us (47813us total) trans 4294972500 rc 0/0 00000100:00100000:3.0:1789760731.159165:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5212 00000100:00100000:3.0:1789760731.159175:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709537339921 time 709577683315 state 0x2 ref 0 00000001:00080000:2.0:1789760731.160303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972500 is committed 00000100:00100000:2.0:1789760731.163149:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.163312:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661312 00000100:00100000:1.0:1789760731.163323:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661312 00000100:00100000:1.0:1789760731.163379:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709537339921 time 709577683315 state 0x0 ref 1 00000100:00100000:1.0:1789760731.163394:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709577943731 time 709577683315 state 0x1 ref 1 00000100:00100000:1.0:1789760731.163426:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709577943731 time 709628715539 state 0x0 ref 1 00000100:00100000:1.0:1789760731.163438:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515661312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.163485:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21233664, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.163505:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163524:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760731.163530:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.163537:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.163552:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.194258:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972501 is committed 00002000:00010000:1.0:1789760731.194364:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21233664 -> 21237760 00010000:00010000:1.0:1789760731.194379:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194392:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194401:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194408:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.194414:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194424:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194443:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194456:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e02a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.194568:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515661312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31127us (31424us total) trans 4294972501 rc 0/0 00000100:00100000:1.0:1789760731.194579:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5213 00000100:00100000:1.0:1789760731.194589:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709577943731 time 709628715539 state 0x2 ref 0 00000100:00100000:1.0:1789760731.199034:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.199140:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661440 00000100:00100000:3.0:1789760731.199150:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661440 00000100:00100000:3.0:1789760731.199207:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709577943731 time 709628715539 state 0x0 ref 1 00000100:00100000:3.0:1789760731.199222:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709628975955 time 709628715539 state 0x1 ref 1 00000100:00100000:3.0:1789760731.199254:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709628975955 time 709664543098 state 0x0 ref 1 00000100:00100000:3.0:1789760731.199266:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515661440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.199318:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21237760, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760731.199327:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.199345:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760731.199349:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.199356:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.199371:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.238661:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21237760 -> 21241856 00010000:00010000:3.0:1789760731.238688:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.238703:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.238738:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.238746:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.238751:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.238760:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.238780:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.238789:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e031 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.238870:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515661440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39601us (39842us total) trans 4294972502 rc 0/0 00000100:00100000:3.0:1789760731.238881:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5214 00000100:00100000:3.0:1789760731.238890:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709628975955 time 709664543098 state 0x2 ref 0 00000001:00080000:2.0:1789760731.238961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972502 is committed 00000100:00100000:3.0:1789760731.242202:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760731.242321:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661568 00000100:00100000:0.0:1789760731.242332:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661568 00000100:00100000:0.0:1789760731.242386:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709628975955 time 709664543098 state 0x0 ref 1 00000100:00100000:0.0:1789760731.242402:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709664803514 time 709664543098 state 0x1 ref 1 00000100:00100000:0.0:1789760731.242465:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709664803514 time 709707754556 state 0x0 ref 1 00000100:00100000:0.0:1789760731.242481:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515661568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760731.242540:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21241856, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:0.0:1789760731.242548:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.242565:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760731.242569:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.242575:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.242589:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.282326:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972503 is committed 00002000:00010000:1.0:1789760731.282476:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21241856 -> 21245952 00010000:00010000:1.0:1789760731.282492:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.282504:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.282513:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.282520:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.282526:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.282534:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.282558:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.282566:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e038 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.282668:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515661568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40180us (40466us total) trans 4294972503 rc 0/0 00000100:00100000:1.0:1789760731.282686:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5215 00000100:00100000:1.0:1789760731.282696:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709664803514 time 709707754556 state 0x2 ref 0 00000100:00100000:0.0:1789760731.287091:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760731.287195:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661696 00000100:00100000:2.0:1789760731.287205:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661696 00000100:00100000:2.0:1789760731.287259:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709664803514 time 709707754556 state 0x0 ref 1 00000100:00100000:2.0:1789760731.287274:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709708014972 time 709707754556 state 0x1 ref 1 00000100:00100000:2.0:1789760731.287347:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709708014972 time 709752635325 state 0x0 ref 1 00000100:00100000:2.0:1789760731.287361:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515661696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760731.287409:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21245952, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760731.287418:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.287432:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760731.287436:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.287441:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.287455:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.322439:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972504 is committed 00002000:00010000:1.0:1789760731.322508:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21245952 -> 21250048 00010000:00010000:1.0:1789760731.322523:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.322536:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.322546:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.322553:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.322558:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.322574:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.322593:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.322601:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e03f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.322695:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515661696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35332us (35608us total) trans 4294972504 rc 0/0 00000100:00100000:1.0:1789760731.323117:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5216 00000100:00100000:1.0:1789760731.323129:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709708014972 time 709752635325 state 0x2 ref 0 00000100:00100000:1.0:1789760731.327451:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760731.328103:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661824 00000100:00100000:2.0:1789760731.328115:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661824 00000100:00100000:2.0:1789760731.328171:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709708014972 time 709752635325 state 0x0 ref 1 00000100:00100000:2.0:1789760731.328187:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709752895741 time 709752635325 state 0x1 ref 1 00000100:00100000:2.0:1789760731.328215:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709752895741 time 709793504454 state 0x0 ref 1 00000100:00100000:2.0:1789760731.328230:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515661824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760731.328277:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21250048, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760731.328286:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.328301:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760731.328305:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.328311:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.328325:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.375726:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972505 is committed 00002000:00010000:1.0:1789760731.375919:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21250048 -> 21254144 00010000:00010000:1.0:1789760731.375934:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.375947:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.375956:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.375968:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.375973:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.376046:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.376076:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.376084:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e046 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.376213:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515661824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47981us (48766us total) trans 4294972505 rc 0/0 00000100:00100000:1.0:1789760731.376224:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5217 00000100:00100000:1.0:1789760731.376234:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709752895741 time 709793504454 state 0x2 ref 0 00000100:00100000:0.0:1789760731.380579:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.380811:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515661952 00000100:00100000:3.0:1789760731.380821:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515661952 00000100:00100000:3.0:1789760731.380871:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709752895741 time 709793504454 state 0x0 ref 1 00000100:00100000:3.0:1789760731.380887:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709793764870 time 709793504454 state 0x1 ref 1 00000100:00100000:3.0:1789760731.380948:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709793764870 time 709846236701 state 0x0 ref 1 00000100:00100000:3.0:1789760731.380961:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515661952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.381019:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21254144, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760731.381029:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.381044:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760731.381048:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.381053:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.381078:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.408445:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21254144 -> 21258240 00010000:00010000:1.0:1789760731.408464:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.408477:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.408492:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.408500:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.408505:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.408515:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.408536:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.408545:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e04d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.408638:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515661952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27674us (28064us total) trans 4294972506 rc 0/0 00000100:00100000:1.0:1789760731.408649:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5218 00000100:00100000:1.0:1789760731.408659:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709793764870 time 709846236701 state 0x2 ref 0 00000001:00080000:0.0:1789760731.410103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972506 is committed 00000100:00100000:0.0:1789760731.414014:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.417599:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662080 00000100:00100000:1.0:1789760731.417616:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662080 00000100:00100000:1.0:1789760731.417683:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709793764870 time 709846236701 state 0x0 ref 1 00000100:00100000:1.0:1789760731.417701:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709846497117 time 709846236701 state 0x1 ref 1 00000100:00100000:1.0:1789760731.417754:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709846497117 time 709883042884 state 0x0 ref 1 00000100:00100000:1.0:1789760731.417769:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515662080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.417819:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21258240, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.417829:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.417845:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760731.417849:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.417856:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.417870:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456121:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760731.496220:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972507 is committed 00002000:00010000:1.0:1789760731.496334:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21258240 -> 21262336 00010000:00010000:1.0:1789760731.496350:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.496363:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.496374:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.496388:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.496392:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.496401:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.496421:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.496429:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e054 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.496521:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515662080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 78751us (82513us total) trans 4294972507 rc 0/0 00000100:00100000:1.0:1789760731.496531:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5219 00000100:00100000:1.0:1789760731.496541:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709846497117 time 709883042884 state 0x2 ref 0 00000100:00100000:3.0:1789760731.499889:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.500012:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662208 00000100:00100000:1.0:1789760731.500022:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662208 00000100:00100000:1.0:1789760731.500086:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709846497117 time 709883042884 state 0x0 ref 1 00000100:00100000:1.0:1789760731.500101:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709883303300 time 709883042884 state 0x1 ref 1 00000100:00100000:1.0:1789760731.500133:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709883303300 time 709965422083 state 0x0 ref 1 00000100:00100000:1.0:1789760731.500150:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515662208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.500201:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21262336, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.500210:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.500226:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760731.500231:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.500238:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.500254:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.523448:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21262336 -> 21266432 00010000:00010000:1.0:1789760731.523468:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.523481:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.523491:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.523499:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.523504:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.523514:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.523534:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.523543:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e05b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.523638:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515662208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23485us (23753us total) trans 4294972508 rc 0/0 00000100:00100000:1.0:1789760731.523649:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5220 00000100:00100000:1.0:1789760731.523660:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709883303300 time 709965422083 state 0x2 ref 0 00000001:00080000:2.0:1789760731.523746:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972508 is committed 00000100:00100000:0.0:1789760731.527169:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.527258:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662336 00000100:00100000:1.0:1789760731.527268:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662336 00000100:00100000:1.0:1789760731.527319:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709883303300 time 709965422083 state 0x0 ref 1 00000100:00100000:1.0:1789760731.527335:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709965682499 time 709965422083 state 0x1 ref 1 00000100:00100000:1.0:1789760731.527363:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709965682499 time 709992652204 state 0x0 ref 1 00000100:00100000:1.0:1789760731.527376:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515662336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.527426:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21266432, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.527435:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.527451:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760731.527456:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.527462:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.527476:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.554243:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972509 is committed 00002000:00010000:1.0:1789760731.554342:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21266432 -> 21270528 00010000:00010000:1.0:1789760731.554357:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.554370:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.554379:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.554387:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.554392:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.554401:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.554423:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.554431:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e062 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.554526:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515662336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27147us (27361us total) trans 4294972509 rc 0/0 00000100:00100000:1.0:1789760731.554537:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5221 00000100:00100000:1.0:1789760731.554546:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709965682499 time 709992652204 state 0x2 ref 0 00000100:00100000:0.0:1789760731.559386:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760731.559476:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662464 00000100:00100000:2.0:1789760731.559487:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662464 00000100:00100000:2.0:1789760731.559567:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709965682499 time 709992652204 state 0x0 ref 1 00000100:00100000:2.0:1789760731.559582:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709992912620 time 709992652204 state 0x1 ref 1 00000100:00100000:3.0:1789760731.559898:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709992912620 time 710025185090 state 0x0 ref 1 00000100:00100000:3.0:1789760731.559914:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515662464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.559963:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21270528, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760731.559971:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.559998:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.560003:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.560009:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.560021:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.592459:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21270528 -> 21274624 00010000:00010000:1.0:1789760731.592480:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.592494:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.592503:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.592510:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.592515:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.592524:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.592547:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.592554:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e069 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.592653:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515662464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32737us (33271us total) trans 4294972510 rc 0/0 00000100:00100000:1.0:1789760731.592664:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5222 00000100:00100000:1.0:1789760731.592673:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709992912620 time 710025185090 state 0x2 ref 0 00000001:00080000:0.0:1789760731.593401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972510 is committed 00000100:00100000:0.0:1789760731.596559:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760731.596679:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662592 00000100:00100000:2.0:1789760731.596689:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662592 00000100:00100000:2.0:1789760731.597028:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 709992912620 time 710025185090 state 0x0 ref 1 00000100:00100000:2.0:1789760731.597044:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710025445506 time 710025185090 state 0x1 ref 1 00000100:00100000:2.0:1789760731.597108:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710025445506 time 710062396886 state 0x0 ref 1 00000100:00100000:2.0:1789760731.597123:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515662592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760731.597169:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21274624, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:2.0:1789760731.597178:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.597193:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.597197:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.597203:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.597216:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.619346:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972511 is committed 00002000:00010000:3.0:1789760731.619653:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21274624 -> 21278720 00010000:00010000:3.0:1789760731.619670:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.619683:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.619693:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.619701:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.619721:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.619731:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.619750:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.619758:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e070 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.619849:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515662592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22724us (23293us total) trans 4294972511 rc 0/0 00000100:00100000:3.0:1789760731.619860:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5223 00000100:00100000:3.0:1789760731.619870:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710025445506 time 710062396886 state 0x2 ref 0 00000100:00100000:0.0:1789760731.623190:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.623411:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662720 00000100:00100000:3.0:1789760731.623421:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662720 00000100:00100000:3.0:1789760731.623545:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710025445506 time 710062396886 state 0x0 ref 1 00000100:00100000:3.0:1789760731.623561:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710062657302 time 710062396886 state 0x1 ref 1 00000100:00100000:3.0:1789760731.623594:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710062657302 time 710088882940 state 0x0 ref 1 00000100:00100000:3.0:1789760731.623608:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515662720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.623656:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21278720, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760731.623664:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623680:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.623686:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.623693:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.623845:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.645853:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544410240:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.646169:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760731.646231:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760731.646239:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be32a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544409856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.646268:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760731.646288:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be32d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544410368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.646311:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760731.646341:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410240 00000100:00100000:2.0:1789760731.646347:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410240 00000100:00100000:2.0:1789760731.646399:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789760731.646408:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544410240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.646457:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544410240:12345-0@lo:400:kworker.0 Request processed in 50us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760731.646465:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789760731.646487:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410368 00000100:00100000:2.0:1789760731.646490:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410368 00000100:00100000:2.0:1789760731.646515:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:2.0:1789760731.646522:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544410368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.646555:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544410368:12345-0@lo:400:kworker.0 Request processed in 34us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760731.646561:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00080000:2.0:1789760731.646576:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760731.646580:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760731.646764:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544410112:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.646796:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.646820:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789760731.646825:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544409984:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.646847:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.646866:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544410496:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647022:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.647053:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409856 00000100:00100000:3.0:1789760731.647066:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409856 00000100:00100000:3.0:1789760731.647102:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:3.0:1789760731.647110:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909353c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544409856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647144:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909353c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544409856:12345-0@lo:400:kworker.0 Request processed in 35us (875us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.647151:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:3.0:1789760731.647182:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410112 00000100:00100000:3.0:1789760731.647186:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410112 00000100:00100000:3.0:1789760731.647210:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544409984 00000100:00100000:3.0:1789760731.647213:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544409984 00000100:00100000:3.0:1789760731.647231:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1789760731.647237:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544410112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647274:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544410112:12345-0@lo:400:kworker.0 Request processed in 37us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.647281:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1789760731.647293:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789760731.647299:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544409984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647325:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544409984:12345-0@lo:400:kworker.0 Request processed in 27us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.647332:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789760731.647351:0:8226:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410496 00000100:00100000:3.0:1789760731.647354:0:8226:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410496 00000100:00100000:3.0:1789760731.647379:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:3.0:1789760731.647385:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544410496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647414:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544410496:12345-0@lo:400:kworker.0 Request processed in 29us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.647421:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00080000:3.0:1789760731.647448:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760731.647452:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760731.647478:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544409984:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647489:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544410496:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.647498:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544410112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.648339:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be32a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544409856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.648354:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be32d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544410368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.648365:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544410240:0@lo:400:kworker.0 00000001:00080000:1.0:1789760731.655614:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972512 is committed 00002000:00010000:3.0:1789760731.655776:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21278720 -> 21282816 00010000:00010000:3.0:1789760731.655790:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655802:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655813:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655821:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.655827:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655837:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655857:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.655866:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e077 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.655967:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515662720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32356us (32782us total) trans 4294972512 rc 0/0 00000100:00100000:3.0:1789760731.655987:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5224 00000100:00100000:3.0:1789760731.655997:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710062657302 time 710088882940 state 0x2 ref 0 00000100:00100000:0.0:1789760731.660107:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.660256:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662848 00000100:00100000:1.0:1789760731.660267:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662848 00000100:00100000:1.0:1789760731.660322:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710062657302 time 710088882940 state 0x0 ref 1 00000100:00100000:1.0:1789760731.660337:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710089143356 time 710088882940 state 0x1 ref 1 00000100:00100000:1.0:1789760731.660374:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710089143356 time 710125662845 state 0x0 ref 1 00000100:00100000:1.0:1789760731.660387:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515662848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.660437:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21282816, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.660445:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.660467:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.660471:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.660477:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.660492:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.709188:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972513 is committed 00002000:00010000:1.0:1789760731.709465:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21282816 -> 21286912 00010000:00010000:1.0:1789760731.709481:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.709494:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.709503:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.709510:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.709515:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.709524:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.709543:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.709552:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e07e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.709960:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515662848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49570us (49857us total) trans 4294972513 rc 0/0 00000100:00100000:1.0:1789760731.709972:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5225 00000100:00100000:1.0:1789760731.709982:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710089143356 time 710125662845 state 0x2 ref 0 00000100:00100000:0.0:1789760731.713609:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.713735:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515662976 00000100:00100000:1.0:1789760731.713744:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515662976 00000100:00100000:1.0:1789760731.713794:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710089143356 time 710125662845 state 0x0 ref 1 00000100:00100000:1.0:1789760731.713809:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710125923261 time 710125662845 state 0x1 ref 1 00000100:00100000:1.0:1789760731.713838:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710125923261 time 710179127832 state 0x0 ref 1 00000100:00100000:1.0:1789760731.713849:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515662976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.713909:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21286912, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.713918:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713934:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.713943:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.713950:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.713969:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.751214:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972514 is committed 00002000:00010000:0.0:1789760731.751737:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21286912 -> 21291008 00010000:00010000:0.0:1789760731.751753:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.751766:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.751775:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.751783:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760731.751788:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.751797:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.751815:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.751823:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e085 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.751917:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515662976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38063us (38313us total) trans 4294972514 rc 0/0 00000100:00100000:0.0:1789760731.751929:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5226 00000100:00100000:0.0:1789760731.751939:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710125923261 time 710179127832 state 0x2 ref 0 00000100:00100000:0.0:1789760731.756288:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.756372:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663104 00000100:00100000:1.0:1789760731.756382:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663104 00000100:00100000:1.0:1789760731.756506:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710125923261 time 710179127832 state 0x0 ref 1 00000100:00100000:1.0:1789760731.756520:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710179388248 time 710179127832 state 0x1 ref 1 00000100:00100000:1.0:1789760731.756550:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710179388248 time 710221840176 state 0x0 ref 1 00000100:00100000:1.0:1789760731.756562:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515663104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.756611:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21291008, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:1.0:1789760731.756622:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.756637:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.756641:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.756648:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.756661:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.791475:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21291008 -> 21295104 00010000:00010000:1.0:1789760731.791497:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.791510:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.791520:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.791527:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760731.791532:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.791541:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.791562:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.791569:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e08c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.791653:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515663104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35087us (35371us total) trans 4294972515 rc 0/0 00000100:00100000:1.0:1789760731.791664:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5227 00000100:00100000:1.0:1789760731.791675:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710179388248 time 710221840176 state 0x2 ref 0 00000001:00080000:2.0:1789760731.792752:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972515 is committed 00000100:00100000:2.0:1789760731.796871:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.796943:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663232 00000100:00100000:3.0:1789760731.796953:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663232 00000100:00100000:3.0:1789760731.797007:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710179388248 time 710221840176 state 0x0 ref 1 00000100:00100000:3.0:1789760731.797025:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710222100592 time 710221840176 state 0x1 ref 1 00000100:00100000:3.0:1789760731.797067:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710222100592 time 710262356222 state 0x0 ref 1 00000100:00100000:3.0:1789760731.797081:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515663232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.797126:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21295104, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760731.797135:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.797150:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.797154:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.797160:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.797173:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.829189:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21295104 -> 21299200 00010000:00010000:3.0:1789760731.829210:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.829223:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.829232:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.829247:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.829252:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.829261:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.829281:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.829289:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e093 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.829377:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515663232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32294us (32512us total) trans 4294972516 rc 0/0 00000100:00100000:3.0:1789760731.829388:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5228 00000100:00100000:3.0:1789760731.829398:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710222100592 time 710262356222 state 0x2 ref 0 00000001:00080000:1.0:1789760731.829471:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972516 is committed 00000100:00100000:0.0:1789760731.832733:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.832829:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663360 00000100:00100000:3.0:1789760731.832838:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663360 00000100:00100000:3.0:1789760731.832889:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710222100592 time 710262356222 state 0x0 ref 1 00000100:00100000:3.0:1789760731.832905:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710262616638 time 710262356222 state 0x1 ref 1 00000100:00100000:3.0:1789760731.832924:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710262616638 time 710298214137 state 0x0 ref 1 00000100:00100000:3.0:1789760731.832939:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515663360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.832994:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21299200, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa807 00010000:00010000:3.0:1789760731.833002:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.833017:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.833021:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.833026:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.833039:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.857902:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972517 is committed 00002000:00010000:3.0:1789760731.858173:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21299200 -> 21303296 00002000:00010000:3.0:1789760731.858178:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760731 -> 1789760732 00002000:00010000:3.0:1789760731.858181:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760731 -> 1789760732 00010000:00010000:3.0:1789760731.858193:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.858206:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.858215:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.858222:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.858227:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.858236:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.858255:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.858261:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e09a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.858383:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515663360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25440us (25687us total) trans 4294972517 rc 0/0 00000100:00100000:3.0:1789760731.858392:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5229 00000100:00100000:3.0:1789760731.858401:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710262616638 time 710298214137 state 0x2 ref 0 00000100:00100000:0.0:1789760731.861969:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760731.862252:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663488 00000100:00100000:3.0:1789760731.862262:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663488 00000100:00100000:3.0:1789760731.862315:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710262616638 time 710298214137 state 0x0 ref 1 00000100:00100000:3.0:1789760731.862330:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710298474553 time 710298214137 state 0x1 ref 1 00000100:00100000:3.0:1789760731.862362:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710298474553 time 710327650784 state 0x0 ref 1 00000100:00100000:3.0:1789760731.862374:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515663488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760731.862417:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21303296, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760731.862427:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.862442:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.862446:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.862453:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.862467:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.892508:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972518 is committed 00002000:00010000:3.0:1789760731.892682:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21303296 -> 21307392 00010000:00010000:3.0:1789760731.892722:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.892737:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.892746:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.892754:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760731.892759:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.892769:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.892796:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.892804:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.892889:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515663488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30512us (30925us total) trans 4294972518 rc 0/0 00000100:00100000:3.0:1789760731.892899:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5230 00000100:00100000:3.0:1789760731.892908:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710298474553 time 710327650784 state 0x2 ref 0 00000100:00100000:2.0:1789760731.895807:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760731.895960:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663616 00000100:00100000:1.0:1789760731.895970:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663616 00000100:00100000:1.0:1789760731.896031:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710298474553 time 710327650784 state 0x0 ref 1 00000100:00100000:1.0:1789760731.896047:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710327911200 time 710327650784 state 0x1 ref 1 00000100:00100000:1.0:1789760731.896082:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710327911200 time 710361371208 state 0x0 ref 1 00000100:00100000:1.0:1789760731.896095:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515663616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760731.896141:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21307392, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:1.0:1789760731.896149:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.896165:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.896169:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.896175:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.896189:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.953593:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21307392 -> 21311488 00010000:00010000:0.0:1789760731.953612:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953626:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953635:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953643:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760731.953648:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953657:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953677:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.953689:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.953807:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515663616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57709us (58004us total) trans 4294972519 rc 0/0 00000100:00100000:0.0:1789760731.953817:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5231 00000100:00100000:0.0:1789760731.953827:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710327911200 time 710361371208 state 0x2 ref 0 00000001:00080000:0.0:1789760731.954859:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972519 is committed 00000100:00100000:0.0:1789760731.958317:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760731.958449:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663744 00000100:00100000:2.0:1789760731.958460:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663744 00000100:00100000:2.0:1789760731.958512:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710327911200 time 710361371208 state 0x0 ref 1 00000100:00100000:2.0:1789760731.958529:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710361631624 time 710361371208 state 0x1 ref 1 00000100:00100000:2.0:1789760731.958756:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710361631624 time 710424042961 state 0x0 ref 1 00000100:00100000:2.0:1789760731.958769:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515663744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760731.958819:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21311488, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:2.0:1789760731.958828:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.958843:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.958848:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.958854:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.958869:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.015504:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972520 is committed 00002000:00010000:2.0:1789760732.015512:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21311488 -> 21315584 00010000:00010000:2.0:1789760732.015527:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.015541:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.015550:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.015557:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760732.015571:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.015581:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.015602:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.015609:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e0af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.015704:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515663744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56932us (57390us total) trans 4294972520 rc 0/0 00000100:00100000:2.0:1789760732.015751:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5232 00000100:00100000:2.0:1789760732.015760:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710361631624 time 710424042961 state 0x2 ref 0 00000100:00100000:0.0:1789760732.021468:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760732.021543:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515663872 00000100:00100000:1.0:1789760732.021554:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515663872 00000100:00100000:1.0:1789760732.021626:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710361631624 time 710424042961 state 0x0 ref 1 00000100:00100000:1.0:1789760732.021640:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710424303377 time 710424042961 state 0x1 ref 1 00000100:00100000:1.0:1789760732.021669:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710424303377 time 710486958491 state 0x0 ref 1 00000100:00100000:1.0:1789760732.021681:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515663872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760732.021752:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21315584, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:1.0:1789760732.021761:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.021777:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760732.021782:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.021789:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760732.021789:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760732.021802:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.085802:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760732.085835:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664128 00000100:00100000:0.0:1789760732.085842:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760732.085844:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664128 00000100:00100000:2.0:1789760732.085901:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 446 00000100:00100000:0.0:1789760732.085910:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760732.085911:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515664128:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760732.085929:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664256 00000100:00100000:0.0:1789760732.085949:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664256 00000100:00100000:0.0:1789760732.085976:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 447 00000100:00100000:0.0:1789760732.085983:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2365:x1876699515664256:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760732.085984:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515664128:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 76us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760732.085991:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 446 00000100:00100000:0.0:1789760732.086009:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805636680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2365:x1876699515664256:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 27us (98us total) trans 0 rc 0/0 00000100:00080000:2.0:1789760732.086011:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760732.086016:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 447 00000100:00100000:1.0:1789760732.086039:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664000 00000100:00100000:1.0:1789760732.086045:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664000 00000100:00100000:1.0:1789760732.086215:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 388 00000100:00100000:1.0:1789760732.086227:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909353050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515664000:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760732.086280:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909353050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515664000:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 56us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760732.086287:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 388 00000001:00080000:1.0:1789760732.102428:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972521 is committed 00002000:00010000:3.0:1789760732.102548:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21315584 -> 21319680 00010000:00010000:3.0:1789760732.102564:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.102577:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.102586:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.102593:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.102599:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.102614:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.102639:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.102647:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e0b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.103069:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515663872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 81375us (81595us total) trans 4294972521 rc 0/0 00000100:00100000:3.0:1789760732.103081:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5233 00000100:00100000:3.0:1789760732.103099:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710424303377 time 710486958491 state 0x2 ref 0 00000100:00100000:0.0:1789760732.110377:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.110467:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664384 00000100:00100000:3.0:1789760732.110477:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664384 00000100:00100000:3.0:1789760732.110525:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710424303377 time 710486958491 state 0x0 ref 1 00000100:00100000:3.0:1789760732.110541:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710487218907 time 710486958491 state 0x1 ref 1 00000100:00100000:3.0:1789760732.110571:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710487218907 time 710575860120 state 0x0 ref 1 00000100:00100000:3.0:1789760732.110583:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515664384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.110630:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21319680, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.110639:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.110658:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760732.110662:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.110669:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.110684:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.151475:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972522 is committed 00002000:00010000:3.0:1789760732.151790:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21319680 -> 21323776 00010000:00010000:3.0:1789760732.151806:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.151819:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.151828:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.151836:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.151840:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.151849:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.151868:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.151877:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.152500:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515664384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41911us (42125us total) trans 4294972522 rc 0/0 00000100:00100000:3.0:1789760732.152511:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5234 00000100:00100000:3.0:1789760732.152521:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710487218907 time 710575860120 state 0x2 ref 0 00000100:00100000:0.0:1789760732.155476:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.155616:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664512 00000100:00100000:3.0:1789760732.155626:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664512 00000100:00100000:3.0:1789760732.155679:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710487218907 time 710575860120 state 0x0 ref 1 00000100:00100000:3.0:1789760732.155695:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710576120536 time 710575860120 state 0x1 ref 1 00000100:00100000:3.0:1789760732.155755:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710576120536 time 710621043609 state 0x0 ref 1 00000100:00100000:3.0:1789760732.155769:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515664512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.155928:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21323776, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.155938:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.155954:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760732.155959:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.155966:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.156061:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.196567:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21323776 -> 21327872 00010000:00010000:3.0:1789760732.196586:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.196600:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.196609:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.196617:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.196622:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.196632:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.196651:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.196660:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e0c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.196773:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515664512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41001us (41301us total) trans 4294972523 rc 0/0 00000100:00100000:3.0:1789760732.196784:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5235 00000100:00100000:3.0:1789760732.196793:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710576120536 time 710621043609 state 0x2 ref 0 00000001:00080000:3.0:1789760732.196827:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972523 is committed 00000100:00100000:2.0:1789760732.199681:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.199751:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664640 00000100:00100000:3.0:1789760732.199761:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664640 00000100:00100000:3.0:1789760732.199814:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710576120536 time 710621043609 state 0x0 ref 1 00000100:00100000:3.0:1789760732.199829:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710621304025 time 710621043609 state 0x1 ref 1 00000100:00100000:3.0:1789760732.199861:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710621304025 time 710665150034 state 0x0 ref 1 00000100:00100000:3.0:1789760732.199873:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515664640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.199922:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21327872, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.199931:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.199946:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760732.199950:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.199956:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.199969:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.263852:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972524 is committed 00002000:00010000:3.0:1789760732.264018:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21327872 -> 21331968 00010000:00010000:3.0:1789760732.264034:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.264064:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.264075:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.264082:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.264209:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.264221:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.264240:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.264248:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.264341:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515664640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64465us (64664us total) trans 4294972524 rc 0/0 00000100:00100000:3.0:1789760732.264376:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5236 00000100:00100000:3.0:1789760732.264385:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710621304025 time 710665150034 state 0x2 ref 0 00000100:00100000:3.0:1789760732.272492:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760732.272646:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664768 00000100:00100000:0.0:1789760732.272656:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664768 00000100:00100000:0.0:1789760732.272742:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710621304025 time 710665150034 state 0x0 ref 1 00000100:00100000:0.0:1789760732.272758:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710665410450 time 710665150034 state 0x1 ref 1 00000100:00100000:0.0:1789760732.272867:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710665410450 time 710738155370 state 0x0 ref 1 00000100:00100000:0.0:1789760732.272879:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515664768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760732.272928:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21331968, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:0.0:1789760732.272936:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.272952:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760732.272956:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.272962:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.272975:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.299454:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21331968 -> 21336064 00010000:00010000:0.0:1789760732.299472:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.299484:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.299497:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.299504:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760732.299508:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.299516:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.299535:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.299542:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e0d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.299625:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515664768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26743us (27136us total) trans 4294972525 rc 0/0 00000100:00100000:0.0:1789760732.299635:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5237 00000100:00100000:0.0:1789760732.299644:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710665410450 time 710738155370 state 0x2 ref 0 00000001:00080000:0.0:1789760732.299683:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972525 is committed 00000100:00100000:1.0:1789760732.305180:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760732.306217:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515664896 00000100:00100000:2.0:1789760732.306229:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515664896 00000100:00100000:2.0:1789760732.306281:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710665410450 time 710738155370 state 0x0 ref 1 00000100:00100000:2.0:1789760732.306296:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710738415786 time 710738155370 state 0x1 ref 1 00000100:00100000:2.0:1789760732.306329:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710738415786 time 710771619121 state 0x0 ref 1 00000100:00100000:2.0:1789760732.306341:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515664896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760732.306388:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21336064, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:2.0:1789760732.306396:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.306411:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760732.306415:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.306421:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.306434:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.366596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972526 is committed 00002000:00010000:3.0:1789760732.369470:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21336064 -> 21340160 00010000:00010000:3.0:1789760732.369489:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.369502:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.369513:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.369522:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.369527:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.369537:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.369557:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.369565:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e0d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.369672:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515664896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63327us (64496us total) trans 4294972526 rc 0/0 00000100:00100000:3.0:1789760732.369683:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5238 00000100:00100000:3.0:1789760732.369693:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710738415786 time 710771619121 state 0x2 ref 0 00000100:00100000:0.0:1789760732.384580:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760732.384648:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665024 00000100:00100000:1.0:1789760732.384658:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665024 00000100:00100000:1.0:1789760732.384756:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710738415786 time 710771619121 state 0x0 ref 1 00000100:00100000:1.0:1789760732.384773:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710771879537 time 710771619121 state 0x1 ref 1 00000100:00100000:1.0:1789760732.384804:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710771879537 time 710850092746 state 0x0 ref 1 00000100:00100000:1.0:1789760732.384817:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760732.384865:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21340160, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:1.0:1789760732.384874:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.384890:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760732.384896:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.384902:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.384916:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.440280:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972527 is committed 00002000:00010000:3.0:1789760732.442920:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21340160 -> 21344256 00010000:00010000:3.0:1789760732.442940:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.442953:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.442964:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.443040:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.443046:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.443064:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.443085:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.443098:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e0e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.443246:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58426us (58670us total) trans 4294972527 rc 0/0 00000100:00100000:3.0:1789760732.443257:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5239 00000100:00100000:3.0:1789760732.443267:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710771879537 time 710850092746 state 0x2 ref 0 00000100:00100000:0.0:1789760732.453971:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.454087:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665152 00000100:00100000:3.0:1789760732.454097:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665152 00000100:00100000:3.0:1789760732.454150:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710771879537 time 710850092746 state 0x0 ref 1 00000100:00100000:3.0:1789760732.454165:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710850353162 time 710850092746 state 0x1 ref 1 00000100:00100000:3.0:1789760732.454226:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710850353162 time 710919515100 state 0x0 ref 1 00000100:00100000:3.0:1789760732.454252:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.454554:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21344256, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.454749:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.454765:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760732.454770:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.454776:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.454796:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.478174:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760732.521459:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21344256 -> 21348352 00010000:00010000:3.0:1789760732.521478:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.521490:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.521528:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.521536:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.521541:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.521550:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.521569:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.521576:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.521655:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515665152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67410us (67689us total) trans 4294972528 rc 0/0 00000100:00100000:3.0:1789760732.521665:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5240 00000100:00100000:3.0:1789760732.521675:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710850353162 time 710919515100 state 0x2 ref 0 00000001:00080000:0.0:1789760732.522375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972528 is committed 00000100:00100000:1.0:1789760732.526006:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.526102:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665280 00000100:00100000:3.0:1789760732.526112:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665280 00000100:00100000:3.0:1789760732.526163:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710850353162 time 710919515100 state 0x0 ref 1 00000100:00100000:3.0:1789760732.526179:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710919775516 time 710919515100 state 0x1 ref 1 00000100:00100000:3.0:1789760732.526211:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710919775516 time 710991500290 state 0x0 ref 1 00000100:00100000:3.0:1789760732.526223:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.526270:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21348352, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.526279:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.526295:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760732.526300:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.526307:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.526321:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.578104:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972529 is committed 00002000:00010000:3.0:1789760732.578214:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21348352 -> 21352448 00010000:00010000:3.0:1789760732.578230:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.578242:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.578310:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.578319:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.578324:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.578334:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.578356:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.578366:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e0ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.578460:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52234us (52458us total) trans 4294972529 rc 0/0 00000100:00100000:3.0:1789760732.578474:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5241 00000100:00100000:3.0:1789760732.578484:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710919775516 time 710991500290 state 0x2 ref 0 00000100:00100000:0.0:1789760732.582139:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.582226:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665408 00000100:00100000:3.0:1789760732.582236:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665408 00000100:00100000:3.0:1789760732.582287:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710919775516 time 710991500290 state 0x0 ref 1 00000100:00100000:3.0:1789760732.582304:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710991760706 time 710991500290 state 0x1 ref 1 00000100:00100000:3.0:1789760732.582368:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710991760706 time 711047656724 state 0x0 ref 1 00000100:00100000:3.0:1789760732.582380:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.582430:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21352448, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.582440:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.582456:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.582460:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.582467:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.582483:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.665389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972530 is committed 00002000:00010000:0.0:1789760732.665524:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21352448 -> 21356544 00010000:00010000:0.0:1789760732.665538:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.665550:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.665559:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.665566:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760732.665571:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.665580:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.665599:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.665607:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e0f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.665738:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 83326us (83574us total) trans 4294972530 rc 0/0 00000100:00100000:0.0:1789760732.665752:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5242 00000100:00100000:0.0:1789760732.665762:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710991760706 time 711047656724 state 0x2 ref 0 00000100:00100000:0.0:1789760732.670296:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760732.670431:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665536 00000100:00100000:1.0:1789760732.670443:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665536 00000100:00100000:1.0:1789760732.670495:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 710991760706 time 711047656724 state 0x0 ref 1 00000100:00100000:1.0:1789760732.670510:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711047917140 time 711047656724 state 0x1 ref 1 00000100:00100000:1.0:1789760732.670540:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711047917140 time 711135829174 state 0x0 ref 1 00000100:00100000:1.0:1789760732.670553:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760732.670599:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21356544, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:1.0:1789760732.670608:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.670623:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.670628:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.670634:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.670649:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.698742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972531 is committed 00002000:00010000:0.0:1789760732.698957:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21356544 -> 21360640 00010000:00010000:0.0:1789760732.699029:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.699041:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.699059:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.699065:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760732.699071:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.699080:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.699099:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.699107:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e0fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.699196:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28641us (28907us total) trans 4294972531 rc 0/0 00000100:00100000:0.0:1789760732.699205:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5243 00000100:00100000:0.0:1789760732.699214:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711047917140 time 711135829174 state 0x2 ref 0 00000100:00100000:2.0:1789760732.703382:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.703463:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665664 00000100:00100000:3.0:1789760732.703472:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665664 00000100:00100000:3.0:1789760732.703523:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711047917140 time 711135829174 state 0x0 ref 1 00000100:00100000:3.0:1789760732.703538:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711136089590 time 711135829174 state 0x1 ref 1 00000100:00100000:3.0:1789760732.703569:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711136089590 time 711168857487 state 0x0 ref 1 00000100:00100000:3.0:1789760732.703582:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515665664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.703629:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21360640, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.703638:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.703653:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.703658:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.703664:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.703677:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.740695:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972532 is committed 00002000:00010000:2.0:1789760732.740951:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21360640 -> 21364736 00010000:00010000:2.0:1789760732.741030:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.741043:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.741062:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.741071:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760732.741076:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.741086:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.741106:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.741115:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e103 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.741239:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515665664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37655us (37862us total) trans 4294972532 rc 0/0 00000100:00100000:2.0:1789760732.741250:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5244 00000100:00100000:2.0:1789760732.741261:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711136089590 time 711168857487 state 0x2 ref 0 00000100:00100000:2.0:1789760732.746135:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.746240:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665792 00000100:00100000:3.0:1789760732.746249:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665792 00000100:00100000:3.0:1789760732.746293:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711136089590 time 711168857487 state 0x0 ref 1 00000100:00100000:3.0:1789760732.746307:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711169117903 time 711168857487 state 0x1 ref 1 00000100:00100000:3.0:1789760732.746329:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711169117903 time 711211619213 state 0x0 ref 1 00000100:00100000:3.0:1789760732.746340:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515665792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.746382:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21364736, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.746391:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.746406:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.746410:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.746416:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.746429:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.776272:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972533 is committed 00002000:00010000:0.0:1789760732.776373:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21364736 -> 21368832 00010000:00010000:0.0:1789760732.776389:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.776402:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.776411:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.776419:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760732.776424:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.776433:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.776453:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.776466:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e10a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.776563:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515665792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30218us (30430us total) trans 4294972533 rc 0/0 00000100:00100000:0.0:1789760732.776573:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5245 00000100:00100000:0.0:1789760732.776582:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711169117903 time 711211619213 state 0x2 ref 0 00000100:00100000:0.0:1789760732.781331:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.781419:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515665920 00000100:00100000:3.0:1789760732.781428:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515665920 00000100:00100000:3.0:1789760732.781477:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711169117903 time 711211619213 state 0x0 ref 1 00000100:00100000:3.0:1789760732.781492:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711211879629 time 711211619213 state 0x1 ref 1 00000100:00100000:3.0:1789760732.781548:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711211879629 time 711246836704 state 0x0 ref 1 00000100:00100000:3.0:1789760732.781560:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.781607:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21368832, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.781617:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.781633:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.781639:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.781645:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.781659:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.828521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972534 is committed 00002000:00010000:0.0:1789760732.829183:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21368832 -> 21372928 00010000:00010000:0.0:1789760732.829200:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.829213:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.829221:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.829228:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760732.829233:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.829241:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.829264:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.829272:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e111 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.829370:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515665920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47807us (48043us total) trans 4294972534 rc 0/0 00000100:00100000:0.0:1789760732.829382:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5246 00000100:00100000:0.0:1789760732.829391:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711211879629 time 711246836704 state 0x2 ref 0 00000100:00100000:0.0:1789760732.832936:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.833109:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666048 00000100:00100000:3.0:1789760732.833118:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666048 00000100:00100000:3.0:1789760732.833168:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711211879629 time 711246836704 state 0x0 ref 1 00000100:00100000:3.0:1789760732.833183:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711247097120 time 711246836704 state 0x1 ref 1 00000100:00100000:3.0:1789760732.833213:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711247097120 time 711298501391 state 0x0 ref 1 00000100:00100000:3.0:1789760732.833224:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515666048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.833273:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21372928, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa807 00010000:00010000:3.0:1789760732.833282:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.833297:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.833301:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.833307:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.833321:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.875601:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972535 is committed 00002000:00010000:1.0:1789760732.875942:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21372928 -> 21377024 00002000:00010000:1.0:1789760732.875946:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760732 -> 1789760733 00002000:00010000:1.0:1789760732.875949:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760732 -> 1789760733 00010000:00010000:1.0:1789760732.875963:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.875990:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.875999:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.876006:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760732.876011:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.876020:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.876040:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.876057:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e118 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.876147:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515666048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42919us (43215us total) trans 4294972535 rc 0/0 00000100:00100000:1.0:1789760732.876157:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5247 00000100:00100000:1.0:1789760732.876166:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711247097120 time 711298501391 state 0x2 ref 0 00000100:00100000:0.0:1789760732.884963:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760732.885152:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666176 00000100:00100000:3.0:1789760732.885161:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666176 00000100:00100000:3.0:1789760732.885217:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711247097120 time 711298501391 state 0x0 ref 1 00000100:00100000:3.0:1789760732.885232:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711298761807 time 711298501391 state 0x1 ref 1 00000100:00100000:3.0:1789760732.885262:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711298761807 time 711350550935 state 0x0 ref 1 00000100:00100000:3.0:1789760732.885275:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515666176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760732.885322:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21377024, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760732.885331:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.885346:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.885350:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.885357:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.885370:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.921020:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972536 is committed 00002000:00010000:3.0:1789760732.921168:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21377024 -> 21381120 00010000:00010000:3.0:1789760732.921184:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.921202:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.921211:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.921218:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760732.921223:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.921232:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.921250:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.921258:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e11f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.921379:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515666176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36102us (36421us total) trans 4294972536 rc 0/0 00000100:00100000:3.0:1789760732.921389:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5248 00000100:00100000:3.0:1789760732.921398:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711298761807 time 711350550935 state 0x2 ref 0 00000100:00100000:0.0:1789760732.925608:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760732.925903:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666304 00000100:00100000:1.0:1789760732.925913:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666304 00000100:00100000:1.0:1789760732.926024:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711298761807 time 711350550935 state 0x0 ref 1 00000100:00100000:1.0:1789760732.926041:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711350811351 time 711350550935 state 0x1 ref 1 00000100:00100000:1.0:1789760732.926109:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711350811351 time 711391397062 state 0x0 ref 1 00000100:00100000:1.0:1789760732.926121:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515666304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760732.926168:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21381120, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:1.0:1789760732.926177:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.926193:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.926197:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.926202:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.926215:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.957666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972537 is committed 00002000:00010000:1.0:1789760732.957921:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21381120 -> 21385216 00010000:00010000:1.0:1789760732.957937:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.957949:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.957963:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.958034:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760732.958039:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.958056:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.958076:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.958083:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e126 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.958214:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515666304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32089us (32609us total) trans 4294972537 rc 0/0 00000100:00100000:1.0:1789760732.958224:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5249 00000100:00100000:1.0:1789760732.958235:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711350811351 time 711391397062 state 0x2 ref 0 00000100:00100000:0.0:1789760732.962566:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760732.962661:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666432 00000100:00100000:2.0:1789760732.962669:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666432 00000100:00100000:2.0:1789760732.966169:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711350811351 time 711391397062 state 0x0 ref 1 00000100:00100000:2.0:1789760732.966192:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711391657478 time 711391397062 state 0x1 ref 1 00000100:00100000:2.0:1789760732.966274:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711391657478 time 711431562364 state 0x0 ref 1 00000100:00100000:2.0:1789760732.966296:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515666432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760732.966361:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21385216, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:2.0:1789760732.966371:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.966386:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760732.966391:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.966397:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.966412:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.007329:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21385216 -> 21389312 00010000:00010000:3.0:1789760733.007349:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.007363:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.007371:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.007379:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.007384:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.007393:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.007414:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.007421:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e12d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.007513:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515666432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41223us (45034us total) trans 4294972538 rc 0/0 00000100:00100000:3.0:1789760733.007525:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5250 00000100:00100000:3.0:1789760733.007534:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711391657478 time 711431562364 state 0x2 ref 0 00000001:00080000:0.0:1789760733.008930:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972538 is committed 00000100:00100000:0.0:1789760733.011159:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.011245:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666560 00000100:00100000:3.0:1789760733.011254:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666560 00000100:00100000:3.0:1789760733.011305:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711391657478 time 711431562364 state 0x0 ref 1 00000100:00100000:3.0:1789760733.011321:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711431822780 time 711431562364 state 0x1 ref 1 00000100:00100000:3.0:1789760733.011353:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711431822780 time 711476641662 state 0x0 ref 1 00000100:00100000:3.0:1789760733.011365:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515666560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.011411:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21389312, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.011420:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.011435:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.011439:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.011445:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.011460:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760733.040813:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21389312 -> 21393408 00010000:00010000:0.0:1789760733.040833:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.040846:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.040855:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.040863:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760733.040868:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.040876:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.040894:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.040901:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e134 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.040990:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515666560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29624us (29835us total) trans 4294972539 rc 0/0 00000100:00100000:0.0:1789760733.041000:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5251 00000100:00100000:0.0:1789760733.041008:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711431822780 time 711476641662 state 0x2 ref 0 00000001:00080000:1.0:1789760733.041543:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972539 is committed 00000100:00100000:1.0:1789760733.044488:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760733.044640:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666688 00000100:00100000:0.0:1789760733.044649:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666688 00000100:00100000:0.0:1789760733.044701:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711431822780 time 711476641662 state 0x0 ref 1 00000100:00100000:0.0:1789760733.044740:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711476902078 time 711476641662 state 0x1 ref 1 00000100:00100000:0.0:1789760733.044758:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711476902078 time 711510047560 state 0x0 ref 1 00000100:00100000:0.0:1789760733.044780:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515666688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760733.044821:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21393408, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:0.0:1789760733.044829:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.044845:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760733.044849:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.044854:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.044867:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.077559:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21393408 -> 21397504 00010000:00010000:2.0:1789760733.077579:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.077592:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.077602:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.077609:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760733.077614:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.077623:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.077640:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.077646:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e13b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.077766:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515666688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32991us (33282us total) trans 4294972540 rc 0/0 00000100:00100000:2.0:1789760733.077891:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5252 00000100:00100000:2.0:1789760733.077900:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711476902078 time 711510047560 state 0x2 ref 0 00000001:00080000:3.0:1789760733.079402:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972540 is committed 00000100:00100000:0.0:1789760733.083161:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760733.083332:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666816 00000100:00100000:1.0:1789760733.083342:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666816 00000100:00100000:1.0:1789760733.083395:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711476902078 time 711510047560 state 0x0 ref 1 00000100:00100000:1.0:1789760733.083410:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711510307976 time 711510047560 state 0x1 ref 1 00000100:00100000:1.0:1789760733.083441:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711510307976 time 711548730405 state 0x0 ref 1 00000100:00100000:1.0:1789760733.083453:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515666816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760733.083497:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21397504, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:1.0:1789760733.083506:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.083522:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760733.083526:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.083533:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.083546:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.121648:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21397504 -> 21401600 00010000:00010000:2.0:1789760733.121668:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.121681:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.121702:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.121743:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760733.121748:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.121758:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.121786:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.121795:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e142 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.121877:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515666816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38420us (38720us total) trans 4294972541 rc 0/0 00000100:00100000:2.0:1789760733.121888:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5253 00000100:00100000:2.0:1789760733.121897:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711510307976 time 711548730405 state 0x2 ref 0 00000001:00080000:0.0:1789760733.124007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972541 is committed 00000100:00100000:2.0:1789760733.126357:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760733.126427:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515666944 00000100:00100000:0.0:1789760733.126437:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515666944 00000100:00100000:0.0:1789760733.126490:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711510307976 time 711548730405 state 0x0 ref 1 00000100:00100000:0.0:1789760733.126505:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711548990821 time 711548730405 state 0x1 ref 1 00000100:00100000:0.0:1789760733.126535:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711548990821 time 711591825331 state 0x0 ref 1 00000100:00100000:0.0:1789760733.126547:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515666944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760733.126595:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21401600, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:0.0:1789760733.126604:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.126619:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760733.126624:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.126630:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.126643:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.157296:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972542 is committed 00002000:00010000:0.0:1789760733.157393:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21401600 -> 21405696 00010000:00010000:0.0:1789760733.157419:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.157431:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.157440:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.157447:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760733.157452:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.157461:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.157485:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.157493:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e149 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.157587:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515666944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31038us (31234us total) trans 4294972542 rc 0/0 00000100:00100000:0.0:1789760733.157598:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5254 00000100:00100000:0.0:1789760733.157608:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711548990821 time 711591825331 state 0x2 ref 0 00000100:00100000:1.0:1789760733.162223:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760733.162311:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667072 00000100:00100000:0.0:1789760733.162321:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667072 00000100:00100000:0.0:1789760733.162383:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711548990821 time 711591825331 state 0x0 ref 1 00000100:00100000:0.0:1789760733.162550:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711592085747 time 711591825331 state 0x1 ref 1 00000100:00100000:3.0:1789760733.163077:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711592085747 time 711628362881 state 0x0 ref 1 00000100:00100000:3.0:1789760733.163093:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515667072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.163144:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21405696, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.163154:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.163169:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.163172:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.163178:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.163192:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.198667:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972543 is committed 00002000:00010000:3.0:1789760733.198859:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21405696 -> 21409792 00010000:00010000:3.0:1789760733.198873:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.198886:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.198895:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.198902:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.198907:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.198915:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.198935:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.198942:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e150 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.199130:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515667072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36034us (36911us total) trans 4294972543 rc 0/0 00000100:00100000:3.0:1789760733.199140:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5255 00000100:00100000:3.0:1789760733.199148:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711592085747 time 711628362881 state 0x2 ref 0 00000100:00100000:1.0:1789760733.202509:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.202574:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667200 00000100:00100000:3.0:1789760733.202584:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667200 00000100:00100000:3.0:1789760733.202634:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711592085747 time 711628362881 state 0x0 ref 1 00000100:00100000:3.0:1789760733.202646:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711628623297 time 711628362881 state 0x1 ref 1 00000100:00100000:3.0:1789760733.202736:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711628623297 time 711668024845 state 0x0 ref 1 00000100:00100000:3.0:1789760733.202748:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515667200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.202801:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21409792, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.202810:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.202824:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.202828:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.202834:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.202846:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.237564:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972544 is committed 00002000:00010000:2.0:1789760733.237659:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21409792 -> 21413888 00010000:00010000:2.0:1789760733.237673:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.237841:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.237856:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.237864:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760733.237868:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.237877:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.237895:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.237902:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e157 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.238141:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515667200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35388us (35634us total) trans 4294972544 rc 0/0 00000100:00100000:3.0:1789760733.238151:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5256 00000100:00100000:3.0:1789760733.238160:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711628623297 time 711668024845 state 0x2 ref 0 00000100:00100000:2.0:1789760733.242234:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.242324:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667328 00000100:00100000:3.0:1789760733.242334:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667328 00000100:00100000:3.0:1789760733.242384:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711628623297 time 711668024845 state 0x0 ref 1 00000100:00100000:3.0:1789760733.242467:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711668285261 time 711668024845 state 0x1 ref 1 00000100:00100000:3.0:1789760733.242535:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711668285261 time 711707822514 state 0x0 ref 1 00000100:00100000:3.0:1789760733.242547:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515667328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.242593:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21413888, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.242602:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.242617:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.242621:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.242628:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.242641:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.275217:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21413888 -> 21417984 00010000:00010000:3.0:1789760733.275237:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.275250:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.275259:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.275266:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.275271:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.275282:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.275300:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.275306:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e15e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.275490:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515667328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32940us (33261us total) trans 4294972545 rc 0/0 00000100:00100000:3.0:1789760733.275502:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5257 00000100:00100000:3.0:1789760733.275512:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711668285261 time 711707822514 state 0x2 ref 0 00000001:00080000:2.0:1789760733.275577:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972545 is committed 00000100:00100000:1.0:1789760733.278945:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.279074:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667456 00000100:00100000:3.0:1789760733.279085:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667456 00000100:00100000:3.0:1789760733.279134:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711668285261 time 711707822514 state 0x0 ref 1 00000100:00100000:3.0:1789760733.279149:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711708082930 time 711707822514 state 0x1 ref 1 00000100:00100000:3.0:1789760733.279229:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711708082930 time 711744517187 state 0x0 ref 1 00000100:00100000:3.0:1789760733.279243:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515667456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.279292:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21417984, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.279300:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.279317:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.279321:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.279327:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.279340:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.312755:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972546 is committed 00002000:00010000:3.0:1789760733.312855:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21417984 -> 21422080 00010000:00010000:3.0:1789760733.312870:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.312883:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.312892:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.312899:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.312904:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.312913:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.312934:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.312942:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e165 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.313158:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515667456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33913us (34217us total) trans 4294972546 rc 0/0 00000100:00100000:3.0:1789760733.313172:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5258 00000100:00100000:3.0:1789760733.313183:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711708082930 time 711744517187 state 0x2 ref 0 00000100:00100000:1.0:1789760733.317481:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760733.317654:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667584 00000100:00100000:2.0:1789760733.317664:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667584 00000100:00100000:2.0:1789760733.317760:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711708082930 time 711744517187 state 0x0 ref 1 00000100:00100000:2.0:1789760733.317774:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711744777603 time 711744517187 state 0x1 ref 1 00000100:00100000:2.0:1789760733.317796:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711744777603 time 711783085342 state 0x0 ref 1 00000100:00100000:2.0:1789760733.317808:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515667584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760733.317852:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21422080, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:2.0:1789760733.317860:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.317877:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760733.317881:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.317886:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.317900:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.341529:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972547 is committed 00002000:00010000:3.0:1789760733.341549:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21422080 -> 21426176 00010000:00010000:3.0:1789760733.341564:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.341577:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.341586:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.341593:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.341598:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.341606:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.341626:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.341634:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e16c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.341773:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515667584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23962us (24297us total) trans 4294972547 rc 0/0 00000100:00100000:3.0:1789760733.341782:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5259 00000100:00100000:3.0:1789760733.341792:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711744777603 time 711783085342 state 0x2 ref 0 00000100:00100000:2.0:1789760733.348627:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760733.348785:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667712 00000100:00100000:0.0:1789760733.348796:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667712 00000100:00100000:0.0:1789760733.348854:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711744777603 time 711783085342 state 0x0 ref 1 00000100:00100000:0.0:1789760733.348869:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711783345758 time 711783085342 state 0x1 ref 1 00000100:00100000:0.0:1789760733.348898:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711783345758 time 711814187658 state 0x0 ref 1 00000100:00100000:0.0:1789760733.348914:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515667712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760733.348960:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21426176, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:0.0:1789760733.348978:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.348994:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760733.348999:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.349005:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.349028:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.375217:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972548 is committed 00002000:00010000:1.0:1789760733.375332:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21426176 -> 21430272 00010000:00010000:1.0:1789760733.375478:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.375495:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.375505:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.375639:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760733.375644:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.375655:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.375676:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.375832:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e173 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.376102:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515667712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27185us (27479us total) trans 4294972548 rc 0/0 00000100:00100000:2.0:1789760733.376115:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5260 00000100:00100000:2.0:1789760733.376125:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711783345758 time 711814187658 state 0x2 ref 0 00000100:00100000:0.0:1789760733.380620:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760733.380795:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667840 00000100:00100000:2.0:1789760733.380805:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667840 00000100:00100000:2.0:1789760733.380858:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711783345758 time 711814187658 state 0x0 ref 1 00000100:00100000:2.0:1789760733.380872:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711814448074 time 711814187658 state 0x1 ref 1 00000100:00100000:2.0:1789760733.380897:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711814448074 time 711846186626 state 0x0 ref 1 00000100:00100000:2.0:1789760733.380911:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515667840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760733.380960:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21430272, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:2.0:1789760733.380968:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.380983:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760733.380988:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.380994:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.381009:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.415906:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972549 is committed 00002000:00010000:3.0:1789760733.416020:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21430272 -> 21434368 00010000:00010000:3.0:1789760733.416035:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.416058:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.416068:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.416075:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.416080:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.416089:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.416108:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.416119:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e17a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.416206:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515667840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35293us (35649us total) trans 4294972549 rc 0/0 00000100:00100000:3.0:1789760733.416215:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5261 00000100:00100000:3.0:1789760733.416225:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711814448074 time 711846186626 state 0x2 ref 0 00000100:00100000:2.0:1789760733.422754:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.422849:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515667968 00000100:00100000:3.0:1789760733.422858:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515667968 00000100:00100000:3.0:1789760733.422911:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711814448074 time 711846186626 state 0x0 ref 1 00000100:00100000:3.0:1789760733.422927:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711846447042 time 711846186626 state 0x1 ref 1 00000100:00100000:3.0:1789760733.422949:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711846447042 time 711888238335 state 0x0 ref 1 00000100:00100000:3.0:1789760733.422961:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515667968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.423019:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21434368, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.423027:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.423053:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.423058:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.423064:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.423084:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.455342:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972550 is committed 00002000:00010000:2.0:1789760733.455444:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21434368 -> 21438464 00010000:00010000:2.0:1789760733.455460:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.455472:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.455480:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.455491:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760733.455496:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.455505:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.455523:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.455531:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e181 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.455632:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515667968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32667us (32882us total) trans 4294972550 rc 0/0 00000100:00100000:2.0:1789760733.455650:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5262 00000100:00100000:2.0:1789760733.455660:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711846447042 time 711888238335 state 0x2 ref 0 00000100:00100000:2.0:1789760733.459219:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.459653:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668096 00000100:00100000:3.0:1789760733.459663:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668096 00000100:00100000:3.0:1789760733.459759:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711846447042 time 711888238335 state 0x0 ref 1 00000100:00100000:3.0:1789760733.459774:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711888498751 time 711888238335 state 0x1 ref 1 00000100:00100000:3.0:1789760733.459798:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711888498751 time 711925088124 state 0x0 ref 1 00000100:00100000:3.0:1789760733.459811:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515668096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.459989:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21438464, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.459999:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.460014:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.460018:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.460024:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.460039:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.495439:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972551 is committed 00002000:00010000:3.0:1789760733.495783:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21438464 -> 21442560 00010000:00010000:3.0:1789760733.495800:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.495813:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.495822:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.495829:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.495834:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.495843:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.495863:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.495871:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e188 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.496064:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515668096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36250us (36847us total) trans 4294972551 rc 0/0 00000100:00100000:3.0:1789760733.496079:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5263 00000100:00100000:3.0:1789760733.496088:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711888498751 time 711925088124 state 0x2 ref 0 00010000:00010000:0.0:1789760733.501856:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760733.503302:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760733.503448:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668224 00000100:00100000:1.0:1789760733.503457:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668224 00000100:00100000:1.0:1789760733.503506:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711888498751 time 711925088124 state 0x0 ref 1 00000100:00100000:1.0:1789760733.503521:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711925348540 time 711925088124 state 0x1 ref 1 00000100:00100000:1.0:1789760733.503550:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711925348540 time 711968839591 state 0x0 ref 1 00000100:00100000:1.0:1789760733.503560:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515668224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760733.503603:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21442560, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:1.0:1789760733.503611:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.503626:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760733.503630:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.503636:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.503658:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760733.526747:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21442560 -> 21446656 00010000:00010000:1.0:1789760733.526774:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.526787:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.526796:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.526804:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760733.526810:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.526819:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.526840:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.526848:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e18f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.527030:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515668224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23466us (23732us total) trans 4294972552 rc 0/0 00000100:00100000:1.0:1789760733.527051:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5264 00000100:00100000:1.0:1789760733.527061:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711925348540 time 711968839591 state 0x2 ref 0 00000001:00080000:1.0:1789760733.527101:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972552 is committed 00000100:00100000:2.0:1789760733.530315:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.530540:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668352 00000100:00100000:3.0:1789760733.530550:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668352 00000100:00100000:3.0:1789760733.530598:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711925348540 time 711968839591 state 0x0 ref 1 00000100:00100000:3.0:1789760733.530613:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711969100007 time 711968839591 state 0x1 ref 1 00000100:00100000:3.0:1789760733.530643:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711969100007 time 711995931724 state 0x0 ref 1 00000100:00100000:3.0:1789760733.530655:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515668352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.530701:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21446656, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.530818:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530834:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760733.530838:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.530844:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.530857:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.561845:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972553 is committed 00002000:00010000:2.0:1789760733.562114:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21446656 -> 21450752 00010000:00010000:2.0:1789760733.562130:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.562143:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.562153:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.562161:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760733.562166:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.562175:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.562195:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.562202:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e196 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.562293:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515668352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31635us (31982us total) trans 4294972553 rc 0/0 00000100:00100000:2.0:1789760733.562303:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5265 00000100:00100000:2.0:1789760733.562313:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711969100007 time 711995931724 state 0x2 ref 0 00000100:00100000:2.0:1789760733.570955:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760733.571114:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668480 00000100:00100000:0.0:1789760733.571126:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668480 00000100:00100000:0.0:1789760733.571181:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711969100007 time 711995931724 state 0x0 ref 1 00000100:00100000:0.0:1789760733.571195:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711996192140 time 711995931724 state 0x1 ref 1 00000100:00100000:0.0:1789760733.571226:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711996192140 time 712036515183 state 0x0 ref 1 00000100:00100000:0.0:1789760733.571244:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515668480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760733.571302:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21450752, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:0.0:1789760733.571311:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.571327:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.571339:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.571346:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.571361:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.629056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972554 is committed 00002000:00010000:1.0:1789760733.629128:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21450752 -> 21454848 00010000:00010000:1.0:1789760733.629145:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.629156:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.629166:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.629178:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760733.629183:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.629191:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.629210:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.629217:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e19d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.629310:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515668480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58065us (58359us total) trans 4294972554 rc 0/0 00000100:00100000:1.0:1789760733.629319:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5266 00000100:00100000:1.0:1789760733.629329:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711996192140 time 712036515183 state 0x2 ref 0 00000100:00100000:0.0:1789760733.636835:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.637293:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668608 00000100:00100000:3.0:1789760733.637303:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668608 00000100:00100000:3.0:1789760733.637576:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 711996192140 time 712036515183 state 0x0 ref 1 00000100:00100000:3.0:1789760733.637594:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712036775599 time 712036515183 state 0x1 ref 1 00000100:00100000:3.0:1789760733.637692:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712036775599 time 712102979967 state 0x0 ref 1 00000100:00100000:3.0:1789760733.637745:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515668608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.637797:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21454848, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.637805:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.637820:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.637824:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.637831:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.637846:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.698156:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544410624:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760733.698490:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760733.698654:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410624 00000100:00100000:2.0:1789760733.698662:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410624 00000100:00100000:2.0:1789760733.698754:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1789760733.698765:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544410624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760733.698803:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760733.698850:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544410624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 87us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760733.698857:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00080000:1.0:1789760733.699039:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760733.699085:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544410624:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760733.704447:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c3bdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544410752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760733.704523:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760733.704614:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410752 00000100:00100000:3.0:1789760733.704623:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410752 00000100:00100000:3.0:1789760733.704695:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:3.0:1789760733.704726:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544410752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760733.704751:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3836739584 avail 3833593856 00000100:00080000:1.0:1789760733.704764:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760733.704809:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544410752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760733.704822:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:1.0:1789760733.705014:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c3bdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544410752:0@lo:13:osp-pre-0-0.0 00002000:00010000:0.0:1789760733.714698:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21454848 -> 21458944 00010000:00010000:0.0:1789760733.714815:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.714827:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.714836:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.714843:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760733.714846:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.714860:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.714879:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.714886:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e1a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.715087:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515668608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 77378us (78255us total) trans 4294972555 rc 0/0 00000100:00100000:0.0:1789760733.715097:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5267 00000100:00100000:0.0:1789760733.715106:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712036775599 time 712102979967 state 0x2 ref 0 00000001:00080000:1.0:1789760733.715462:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972555 is committed 00000100:00100000:2.0:1789760733.724876:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.724979:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668736 00000100:00100000:3.0:1789760733.724989:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668736 00000100:00100000:3.0:1789760733.725050:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712036775599 time 712102979967 state 0x0 ref 1 00000100:00100000:3.0:1789760733.725067:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712103240383 time 712102979967 state 0x1 ref 1 00000100:00100000:3.0:1789760733.725105:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712103240383 time 712190390217 state 0x0 ref 1 00000100:00100000:3.0:1789760733.725118:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515668736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.725165:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21458944, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.725173:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.725188:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.725193:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.725199:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.725213:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.748785:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972556 is committed 00002000:00010000:0.0:1789760733.748830:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21458944 -> 21463040 00010000:00010000:0.0:1789760733.748847:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.748861:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.748869:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.748874:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760733.748878:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.748884:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.748900:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.748904:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e1ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.748990:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515668736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23872us (24123us total) trans 4294972556 rc 0/0 00000100:00100000:0.0:1789760733.748998:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5268 00000100:00100000:0.0:1789760733.749005:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712103240383 time 712190390217 state 0x2 ref 0 00000100:00100000:2.0:1789760733.753856:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760733.755732:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668864 00000100:00100000:3.0:1789760733.755746:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668864 00000100:00100000:3.0:1789760733.755805:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712103240383 time 712190390217 state 0x0 ref 1 00000100:00100000:3.0:1789760733.755820:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712190650633 time 712190390217 state 0x1 ref 1 00000100:00100000:3.0:1789760733.755888:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712190650633 time 712221177032 state 0x0 ref 1 00000100:00100000:3.0:1789760733.755903:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515668864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760733.755960:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21463040, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:3.0:1789760733.755970:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.755986:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.755990:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.755997:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.756012:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760733.784476:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21463040 -> 21467136 00010000:00010000:3.0:1789760733.784495:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.784507:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.784516:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.784523:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.784529:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.784539:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.784560:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.784568:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e1b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.784657:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515668864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28753us (30805us total) trans 4294972557 rc 0/0 00000100:00100000:3.0:1789760733.784667:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5269 00000100:00100000:3.0:1789760733.784677:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712190650633 time 712221177032 state 0x2 ref 0 00000001:00080000:1.0:1789760733.785879:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972557 is committed 00000100:00100000:3.0:1789760733.788281:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760733.788396:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515668992 00000100:00100000:1.0:1789760733.788406:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515668992 00000100:00100000:1.0:1789760733.788458:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712190650633 time 712221177032 state 0x0 ref 1 00000100:00100000:1.0:1789760733.788473:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712221437448 time 712221177032 state 0x1 ref 1 00000100:00100000:1.0:1789760733.788552:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712221437448 time 712253840729 state 0x0 ref 1 00000100:00100000:1.0:1789760733.788565:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515668992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760733.788611:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21467136, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:1.0:1789760733.788619:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.788634:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.788638:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.788644:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.788658:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.834312:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972558 is committed 00002000:00010000:1.0:1789760733.834625:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21467136 -> 21471232 00010000:00010000:1.0:1789760733.834642:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.834654:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.834663:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.834670:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760733.834675:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.834694:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.834736:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.834744:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.834868:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515668992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46299us (46592us total) trans 4294972558 rc 0/0 00000100:00100000:1.0:1789760733.834879:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5270 00000100:00100000:1.0:1789760733.834888:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712221437448 time 712253840729 state 0x2 ref 0 00000100:00100000:3.0:1789760733.838033:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760733.838407:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669120 00000100:00100000:2.0:1789760733.838418:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669120 00000100:00100000:2.0:1789760733.838473:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712221437448 time 712253840729 state 0x0 ref 1 00000100:00100000:2.0:1789760733.838487:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712254101145 time 712253840729 state 0x1 ref 1 00000100:00100000:2.0:1789760733.838527:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712254101145 time 712303811808 state 0x0 ref 1 00000100:00100000:2.0:1789760733.838539:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515669120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760733.838587:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21471232, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa807 00010000:00010000:2.0:1789760733.838596:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.838611:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760733.838615:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.838622:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.838635:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.896672:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972559 is committed 00002000:00010000:3.0:1789760733.896774:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21471232 -> 21475328 00002000:00010000:3.0:1789760733.896779:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760733 -> 1789760734 00002000:00010000:3.0:1789760733.896781:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760733 -> 1789760734 00010000:00010000:3.0:1789760733.896794:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.896808:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.896818:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.896825:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.896829:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.896838:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.896856:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.896864:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1e1c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.896951:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515669120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58408us (58921us total) trans 4294972559 rc 0/0 00000100:00100000:3.0:1789760733.896970:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5271 00000100:00100000:3.0:1789760733.896979:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712254101145 time 712303811808 state 0x2 ref 0 00000100:00100000:2.0:1789760733.904525:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760733.906436:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669248 00000100:00100000:1.0:1789760733.906450:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669248 00000100:00100000:1.0:1789760733.906511:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712254101145 time 712303811808 state 0x0 ref 1 00000100:00100000:1.0:1789760733.906527:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712304072224 time 712303811808 state 0x1 ref 1 00000100:00100000:1.0:1789760733.906561:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712304072224 time 712371850391 state 0x0 ref 1 00000100:00100000:1.0:1789760733.906575:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515669248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760733.906636:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21475328, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:1.0:1789760733.906646:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.906663:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.906668:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.906674:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.906696:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.952492:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972560 is committed 00002000:00010000:3.0:1789760733.952819:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21475328 -> 21479424 00010000:00010000:3.0:1789760733.952834:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.952853:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.952862:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.952971:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760733.952977:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.952986:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.953005:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.953013:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e1c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.953660:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515669248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47079us (49135us total) trans 4294972560 rc 0/0 00000100:00100000:2.0:1789760733.953672:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5272 00000100:00100000:2.0:1789760733.953939:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712304072224 time 712371850391 state 0x2 ref 0 00000100:00100000:0.0:1789760733.962157:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760733.962369:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669376 00000100:00100000:2.0:1789760733.962380:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669376 00000100:00100000:2.0:1789760733.962539:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712304072224 time 712371850391 state 0x0 ref 1 00000100:00100000:2.0:1789760733.962555:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712372110807 time 712371850391 state 0x1 ref 1 00000100:00100000:2.0:1789760733.962585:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712372110807 time 712427874392 state 0x0 ref 1 00000100:00100000:2.0:1789760733.962599:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515669376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760733.962649:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21479424, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:2.0:1789760733.962658:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.962674:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760733.962679:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.962686:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.962700:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.006442:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972561 is committed 00002000:00010000:3.0:1789760734.006595:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21479424 -> 21483520 00010000:00010000:3.0:1789760734.006617:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.006631:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.006646:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.006653:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760734.006664:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.006674:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.006693:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.006702:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e1ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.006827:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515669376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44228us (44676us total) trans 4294972561 rc 0/0 00000100:00100000:3.0:1789760734.006837:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5273 00000100:00100000:3.0:1789760734.006847:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712372110807 time 712427874392 state 0x2 ref 0 00000100:00100000:3.0:1789760734.011235:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760734.011296:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669504 00000100:00100000:0.0:1789760734.011306:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669504 00000100:00100000:0.0:1789760734.011378:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712372110807 time 712427874392 state 0x0 ref 1 00000100:00080000:1.0:1789760734.011462:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760734.011506:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712428134808 time 712427874392 state 0x1 ref 1 00000100:00100000:0.0:1789760734.011537:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712428134808 time 712476826706 state 0x0 ref 1 00000100:00100000:0.0:1789760734.011551:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515669504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760734.011601:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21483520, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:0.0:1789760734.011609:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.011624:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760734.011628:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.011634:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.011647:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.043394:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972562 is committed 00002000:00010000:0.0:1789760734.043529:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21483520 -> 21487616 00010000:00010000:0.0:1789760734.043545:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.043558:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.043566:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.043573:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760734.043578:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.043588:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.043608:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.043615:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.043747:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515669504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32195us (32519us total) trans 4294972562 rc 0/0 00000100:00100000:0.0:1789760734.043757:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5274 00000100:00100000:0.0:1789760734.043766:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712428134808 time 712476826706 state 0x2 ref 0 00000100:00100000:1.0:1789760734.052612:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760734.052834:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669632 00000100:00100000:2.0:1789760734.052844:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669632 00000100:00100000:2.0:1789760734.052964:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712428134808 time 712476826706 state 0x0 ref 1 00000100:00100000:2.0:1789760734.052979:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712477087122 time 712476826706 state 0x1 ref 1 00000100:00100000:2.0:1789760734.053009:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712477087122 time 712518298329 state 0x0 ref 1 00000100:00100000:2.0:1789760734.053022:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515669632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760734.053181:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21487616, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:2.0:1789760734.053190:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.053205:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760734.053208:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.053214:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.053227:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.111851:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972563 is committed 00002000:00010000:3.0:1789760734.112023:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21487616 -> 21491712 00010000:00010000:3.0:1789760734.112194:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.112208:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.112217:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.112224:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760734.112229:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.112239:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.112259:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.112267:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e1dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.112448:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515669632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59423us (59840us total) trans 4294972563 rc 0/0 00000100:00100000:3.0:1789760734.112459:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5275 00000100:00100000:3.0:1789760734.112469:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712477087122 time 712518298329 state 0x2 ref 0 00000100:00100000:0.0:1789760734.119287:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760734.119327:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669760 00000100:00100000:1.0:1789760734.119336:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669760 00000100:00100000:1.0:1789760734.119392:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712477087122 time 712518298329 state 0x0 ref 1 00000100:00100000:1.0:1789760734.119408:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712518558745 time 712518298329 state 0x1 ref 1 00000100:00100000:1.0:1789760734.119432:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712518558745 time 712584721400 state 0x0 ref 1 00000100:00100000:1.0:1789760734.119443:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515669760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760734.119560:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21491712, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:1.0:1789760734.119570:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.119586:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760734.119591:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.119598:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.119611:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.176007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972564 is committed 00002000:00010000:1.0:1789760734.176190:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21491712 -> 21495808 00010000:00010000:1.0:1789760734.176206:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.176219:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.176229:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.176237:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760734.176242:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.176252:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.176273:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.176282:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.176375:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515669760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56928us (58942us total) trans 4294972564 rc 0/0 00000100:00100000:1.0:1789760734.176386:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5276 00000100:00100000:1.0:1789760734.176396:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712518558745 time 712584721400 state 0x2 ref 0 00000100:00100000:3.0:1789760734.179361:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760734.182327:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515669888 00000100:00100000:0.0:1789760734.182341:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515669888 00000100:00100000:0.0:1789760734.182400:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712518558745 time 712584721400 state 0x0 ref 1 00000100:00100000:0.0:1789760734.182416:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712584981816 time 712584721400 state 0x1 ref 1 00000100:00100000:0.0:1789760734.182451:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712584981816 time 712647740685 state 0x0 ref 1 00000100:00100000:0.0:1789760734.182465:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515669888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760734.182510:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21495808, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:0.0:1789760734.182519:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.182539:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760734.182544:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.182550:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.182563:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.236745:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972565 is committed 00002000:00010000:0.0:1789760734.239217:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21495808 -> 21499904 00010000:00010000:0.0:1789760734.239236:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.239250:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.239259:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.239268:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760734.239273:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.239282:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.239302:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.239310:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e1ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.239428:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515669888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56960us (60071us total) trans 4294972565 rc 0/0 00000100:00100000:0.0:1789760734.239439:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5277 00000100:00100000:0.0:1789760734.239450:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712584981816 time 712647740685 state 0x2 ref 0 00000100:00100000:2.0:1789760734.248839:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760734.249146:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670016 00000100:00100000:1.0:1789760734.249155:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670016 00000100:00100000:1.0:1789760734.249207:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712584981816 time 712647740685 state 0x0 ref 1 00000100:00100000:1.0:1789760734.249222:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712648001101 time 712647740685 state 0x1 ref 1 00000100:00100000:1.0:1789760734.252060:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712648001101 time 712717327717 state 0x0 ref 1 00000100:00100000:1.0:1789760734.252492:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760734.252549:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21499904, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:1.0:1789760734.252563:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.252580:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760734.252584:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.252590:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.252603:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.305805:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972566 is committed 00002000:00010000:2.0:1789760734.305954:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21499904 -> 21504000 00010000:00010000:2.0:1789760734.305977:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.305994:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.306004:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.306011:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760734.306016:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.306024:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.306053:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.306062:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e1f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.306152:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53670us (57316us total) trans 4294972566 rc 0/0 00000100:00100000:2.0:1789760734.306162:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5278 00000100:00100000:2.0:1789760734.306172:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712648001101 time 712717327717 state 0x2 ref 0 00000100:00100000:3.0:1789760734.315157:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760734.315452:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670144 00000100:00100000:2.0:1789760734.315463:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670144 00000100:00100000:2.0:1789760734.315535:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712648001101 time 712717327717 state 0x0 ref 1 00000100:00100000:2.0:1789760734.315551:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712717588133 time 712717327717 state 0x1 ref 1 00000100:00100000:2.0:1789760734.315595:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712717588133 time 712780884026 state 0x0 ref 1 00000100:00100000:2.0:1789760734.315608:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515670144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760734.315666:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21504000, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:2.0:1789760734.315676:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.315692:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760734.315697:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.315703:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.315753:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.347650:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21504000 -> 21508096 00010000:00010000:3.0:1789760734.347670:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.348185:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.348195:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.348202:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760734.348207:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.348216:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.348236:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.348244:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e1f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.348337:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515670144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32726us (33184us total) trans 4294972567 rc 0/0 00000100:00100000:3.0:1789760734.348348:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5279 00000100:00100000:3.0:1789760734.348358:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712717588133 time 712780884026 state 0x2 ref 0 00000001:00080000:3.0:1789760734.348400:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972567 is committed 00000100:00100000:3.0:1789760734.356893:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760734.357364:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670272 00000100:00100000:2.0:1789760734.357374:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670272 00000100:00100000:2.0:1789760734.357429:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712717588133 time 712780884026 state 0x0 ref 1 00000100:00100000:2.0:1789760734.357445:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712781144442 time 712780884026 state 0x1 ref 1 00000100:00100000:2.0:1789760734.357474:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712781144442 time 712822763426 state 0x0 ref 1 00000100:00100000:2.0:1789760734.357490:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760734.357546:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21508096, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:2.0:1789760734.357554:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.357568:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760734.357572:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.357578:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.357591:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.436361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972568 is committed 00002000:00010000:2.0:1789760734.436554:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21508096 -> 21512192 00010000:00010000:2.0:1789760734.436570:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.436583:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.436592:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.436600:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760734.436605:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.436614:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.436635:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.436643:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e1ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.440418:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 82916us (83520us total) trans 4294972568 rc 0/0 00000100:00100000:2.0:1789760734.440433:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5280 00000100:00100000:2.0:1789760734.440443:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712781144442 time 712822763426 state 0x2 ref 0 00000100:00100000:2.0:1789760734.448824:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760734.448915:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670400 00000100:00100000:3.0:1789760734.448926:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670400 00000100:00100000:3.0:1789760734.448990:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712781144442 time 712822763426 state 0x0 ref 1 00000100:00100000:3.0:1789760734.449005:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712823023842 time 712822763426 state 0x1 ref 1 00000100:00100000:3.0:1789760734.449042:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712823023842 time 712914322694 state 0x0 ref 1 00000100:00100000:3.0:1789760734.449056:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515670400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760734.449104:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21512192, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:3.0:1789760734.449114:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.449130:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760734.449134:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.449141:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.449155:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.496604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972569 is committed 00002000:00010000:1.0:1789760734.498912:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21512192 -> 21516288 00010000:00010000:1.0:1789760734.498932:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.498945:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.499022:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.499147:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760734.499153:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.499162:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.499181:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.499188:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e206 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.499347:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515670400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50288us (50526us total) trans 4294972569 rc 0/0 00000100:00100000:1.0:1789760734.499358:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5281 00000100:00100000:1.0:1789760734.499367:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712823023842 time 712914322694 state 0x2 ref 0 00000100:00100000:0.0:1789760734.512518:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760734.512620:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670528 00000100:00100000:1.0:1789760734.512630:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670528 00000100:00100000:1.0:1789760734.512683:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712823023842 time 712914322694 state 0x0 ref 1 00000100:00100000:1.0:1789760734.512697:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712914583110 time 712914322694 state 0x1 ref 1 00000100:00100000:1.0:1789760734.512757:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712914583110 time 712978045945 state 0x0 ref 1 00000100:00100000:1.0:1789760734.512772:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760734.512819:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21516288, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:1.0:1789760734.512828:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.512842:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760734.512846:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.512853:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.512867:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.526252:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760734.562984:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972570 is committed 00002000:00010000:1.0:1789760734.563278:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21516288 -> 21520384 00010000:00010000:1.0:1789760734.563295:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.563306:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.563314:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.563322:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760734.563326:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.563336:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.563358:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.563366:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e20d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.563490:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50715us (50976us total) trans 4294972570 rc 0/0 00000100:00100000:1.0:1789760734.563501:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5282 00000100:00100000:1.0:1789760734.563510:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712914583110 time 712978045945 state 0x2 ref 0 00000100:00100000:2.0:1789760734.567316:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760734.567514:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670656 00000100:00100000:0.0:1789760734.567523:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670656 00000100:00100000:0.0:1789760734.567573:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712914583110 time 712978045945 state 0x0 ref 1 00000100:00100000:0.0:1789760734.567588:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712978306361 time 712978045945 state 0x1 ref 1 00000100:00100000:0.0:1789760734.567618:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712978306361 time 713032907088 state 0x0 ref 1 00000100:00100000:0.0:1789760734.567630:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515670656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760734.567678:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21520384, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:0.0:1789760734.567697:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.567886:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760734.567891:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.567899:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.567913:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.605019:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21520384 -> 21524480 00010000:00010000:3.0:1789760734.605048:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.605060:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.605069:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.605076:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760734.605081:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.605090:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.605108:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.605116:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e214 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.605204:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515670656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37571us (37892us total) trans 4294972571 rc 0/0 00000100:00100000:3.0:1789760734.605218:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5283 00000100:00100000:3.0:1789760734.605228:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712978306361 time 713032907088 state 0x2 ref 0 00000001:00080000:2.0:1789760734.606644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972571 is committed 00000100:00100000:2.0:1789760734.608794:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760734.610524:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670784 00000100:00100000:0.0:1789760734.610538:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670784 00000100:00100000:0.0:1789760734.610602:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 712978306361 time 713032907088 state 0x0 ref 1 00000100:00100000:0.0:1789760734.610618:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713033167504 time 713032907088 state 0x1 ref 1 00000100:00100000:0.0:1789760734.610655:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713033167504 time 713075943981 state 0x0 ref 1 00000100:00100000:0.0:1789760734.610669:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515670784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760734.610758:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21524480, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:0.0:1789760734.610766:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.610783:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760734.610788:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.610794:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.610810:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.641202:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21524480 -> 21528576 00010000:00010000:1.0:1789760734.641223:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.641236:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.641246:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.641253:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760734.641258:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.641267:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.641286:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.641293:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e21b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.641386:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515670784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30714us (32593us total) trans 4294972572 rc 0/0 00000100:00100000:1.0:1789760734.641396:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5284 00000100:00100000:1.0:1789760734.641406:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713033167504 time 713075943981 state 0x2 ref 0 00000001:00080000:3.0:1789760734.641485:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972572 is committed 00000100:00100000:0.0:1789760734.645618:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760734.645746:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515670912 00000100:00100000:3.0:1789760734.645756:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515670912 00000100:00100000:3.0:1789760734.645807:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713033167504 time 713075943981 state 0x0 ref 1 00000100:00100000:3.0:1789760734.645822:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713076204397 time 713075943981 state 0x1 ref 1 00000100:00100000:3.0:1789760734.645846:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713076204397 time 713111135886 state 0x0 ref 1 00000100:00100000:3.0:1789760734.645858:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515670912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760734.645908:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21528576, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:3.0:1789760734.645917:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.645932:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.645937:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.645943:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.645970:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.677304:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972573 is committed 00002000:00010000:2.0:1789760734.677518:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21528576 -> 21532672 00010000:00010000:2.0:1789760734.677533:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.677546:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.677561:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.677568:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760734.677573:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.677582:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.677600:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.677608:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e222 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.678186:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515670912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32324us (32573us total) trans 4294972573 rc 0/0 00000100:00100000:2.0:1789760734.678196:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5285 00000100:00100000:2.0:1789760734.678205:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713076204397 time 713111135886 state 0x2 ref 0 00000100:00100000:0.0:1789760734.685016:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760734.685421:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671040 00000100:00100000:2.0:1789760734.685431:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671040 00000100:00100000:2.0:1789760734.685544:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713076204397 time 713111135886 state 0x0 ref 1 00000100:00100000:2.0:1789760734.685620:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713111396302 time 713111135886 state 0x1 ref 1 00000100:00100000:2.0:1789760734.685651:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713111396302 time 713150940575 state 0x0 ref 1 00000100:00100000:2.0:1789760734.685664:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515671040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760734.685738:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21532672, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:2.0:1789760734.685748:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.685765:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760734.685770:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.685776:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.685790:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.723922:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972574 is committed 00002000:00010000:2.0:1789760734.724135:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21532672 -> 21536768 00010000:00010000:2.0:1789760734.724152:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.724166:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.724175:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.724183:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760734.724188:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.724198:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.724217:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.724224:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e229 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.724310:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515671040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38644us (39957us total) trans 4294972574 rc 0/0 00000100:00100000:2.0:1789760734.724319:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5286 00000100:00100000:2.0:1789760734.724328:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713111396302 time 713150940575 state 0x2 ref 0 00000100:00100000:1.0:1789760734.727950:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760734.728060:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671168 00000100:00100000:3.0:1789760734.728070:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671168 00000100:00100000:3.0:1789760734.728121:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713111396302 time 713150940575 state 0x0 ref 1 00000100:00100000:3.0:1789760734.728138:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713151200991 time 713150940575 state 0x1 ref 1 00000100:00100000:3.0:1789760734.728169:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713151200991 time 713193458403 state 0x0 ref 1 00000100:00100000:3.0:1789760734.728182:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515671168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760734.728229:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21536768, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:3.0:1789760734.728238:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.728253:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.728258:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.728264:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.728278:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.766464:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972575 is committed 00002000:00010000:3.0:1789760734.766640:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21536768 -> 21540864 00010000:00010000:3.0:1789760734.766656:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.766667:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.766692:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.766698:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760734.766703:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.766746:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.766766:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.766773:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e230 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.766864:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515671168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38679us (38918us total) trans 4294972575 rc 0/0 00000100:00100000:3.0:1789760734.766874:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5287 00000100:00100000:3.0:1789760734.766883:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713151200991 time 713193458403 state 0x2 ref 0 00000100:00100000:2.0:1789760734.771172:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760734.771337:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671296 00000100:00100000:1.0:1789760734.771346:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671296 00000100:00100000:1.0:1789760734.771399:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713151200991 time 713193458403 state 0x0 ref 1 00000100:00100000:1.0:1789760734.771415:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713193718819 time 713193458403 state 0x1 ref 1 00000100:00100000:1.0:1789760734.771444:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713193718819 time 713236733475 state 0x0 ref 1 00000100:00100000:1.0:1789760734.771457:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515671296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760734.771509:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21540864, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:1.0:1789760734.771518:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.771533:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.771537:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.771543:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.771556:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.806791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972576 is committed 00002000:00010000:2.0:1789760734.806963:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21540864 -> 21544960 00010000:00010000:2.0:1789760734.806979:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.806991:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.807001:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.807015:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760734.807020:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.807151:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.807175:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.807185:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e237 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.807283:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515671296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35822us (36114us total) trans 4294972576 rc 0/0 00000100:00100000:2.0:1789760734.807293:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5288 00000100:00100000:2.0:1789760734.807303:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713193718819 time 713236733475 state 0x2 ref 0 00000100:00100000:2.0:1789760734.815848:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760734.816237:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671424 00000100:00100000:3.0:1789760734.816248:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671424 00000100:00100000:3.0:1789760734.816301:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713193718819 time 713236733475 state 0x0 ref 1 00000100:00100000:3.0:1789760734.816319:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713236993891 time 713236733475 state 0x1 ref 1 00000100:00100000:3.0:1789760734.816351:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713236993891 time 713281639430 state 0x0 ref 1 00000100:00100000:3.0:1789760734.816364:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515671424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760734.816585:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21544960, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa807 00010000:00010000:3.0:1789760734.816594:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.816609:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.816614:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.816620:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.816680:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.870065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972577 is committed 00002000:00010000:3.0:1789760734.870282:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21544960 -> 21549056 00002000:00010000:3.0:1789760734.870287:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760734 -> 1789760735 00002000:00010000:3.0:1789760734.870290:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760734 -> 1789760735 00010000:00010000:3.0:1789760734.870305:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.870316:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.870330:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.870338:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760734.870343:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.870417:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.870437:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.870444:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e23e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.870570:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515671424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54204us (54727us total) trans 4294972577 rc 0/0 00000100:00100000:3.0:1789760734.870582:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5289 00000100:00100000:3.0:1789760734.870592:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713236993891 time 713281639430 state 0x2 ref 0 00000100:00100000:0.0:1789760734.876234:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760734.876329:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671552 00000100:00100000:1.0:1789760734.876338:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671552 00000100:00100000:1.0:1789760734.876465:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713236993891 time 713281639430 state 0x0 ref 1 00000100:00100000:1.0:1789760734.876480:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713281899846 time 713281639430 state 0x1 ref 1 00000100:00100000:1.0:1789760734.876510:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713281899846 time 713341799514 state 0x0 ref 1 00000100:00100000:1.0:1789760734.876522:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515671552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760734.876569:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21549056, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:1.0:1789760734.876577:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.876598:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.876603:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.876608:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.876621:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.913644:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21549056 -> 21553152 00010000:00010000:1.0:1789760734.913666:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.913680:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.913691:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.913699:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760734.913704:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.913734:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.913755:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.913775:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e245 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.913870:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515671552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37345us (37639us total) trans 4294972578 rc 0/0 00000100:00100000:1.0:1789760734.913881:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5290 00000100:00100000:1.0:1789760734.913892:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713281899846 time 713341799514 state 0x2 ref 0 00000001:00080000:3.0:1789760734.914576:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972578 is committed 00000100:00100000:2.0:1789760734.928218:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760734.929724:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671680 00000100:00100000:3.0:1789760734.929736:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671680 00000100:00100000:3.0:1789760734.929867:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713281899846 time 713341799514 state 0x0 ref 1 00000100:00100000:3.0:1789760734.929887:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713342059930 time 713341799514 state 0x1 ref 1 00000100:00100000:3.0:1789760734.929977:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713342059930 time 713395264591 state 0x0 ref 1 00000100:00100000:3.0:1789760734.929991:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515671680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760734.930056:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21553152, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760734.930065:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.930079:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.930083:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.930089:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.930103:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.014942:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972579 is committed 00002000:00010000:3.0:1789760735.019829:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21553152 -> 21557248 00010000:00010000:3.0:1789760735.019854:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.019866:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.019876:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.019888:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.019893:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.019902:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.019921:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.019929:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e24c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.020108:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515671680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 90114us (91894us total) trans 4294972579 rc 0/0 00000100:00100000:3.0:1789760735.020119:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5291 00000100:00100000:3.0:1789760735.020129:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713342059930 time 713395264591 state 0x2 ref 0 00000100:00100000:0.0:1789760735.026871:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760735.029156:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671808 00000100:00100000:2.0:1789760735.029170:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671808 00000100:00100000:2.0:1789760735.029232:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713342059930 time 713395264591 state 0x0 ref 1 00000100:00100000:2.0:1789760735.029247:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713395525007 time 713395264591 state 0x1 ref 1 00000100:00100000:2.0:1789760735.029285:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713395525007 time 713494573533 state 0x0 ref 1 00000100:00100000:2.0:1789760735.029298:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515671808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760735.029361:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21557248, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:2.0:1789760735.029370:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.029384:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760735.029389:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.029395:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.029408:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.091151:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21557248 -> 21561344 00010000:00010000:3.0:1789760735.091171:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.091184:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.091199:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.091207:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.091212:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.091221:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.091240:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.091248:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e253 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.091428:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515671808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62127us (64561us total) trans 4294972580 rc 0/0 00000100:00100000:3.0:1789760735.091439:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5292 00000100:00100000:3.0:1789760735.091449:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713395525007 time 713494573533 state 0x2 ref 0 00000001:00080000:1.0:1789760735.094951:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972580 is committed 00000100:00100000:1.0:1789760735.099231:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.099690:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515671936 00000100:00100000:3.0:1789760735.099699:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515671936 00000100:00100000:3.0:1789760735.099778:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713395525007 time 713494573533 state 0x0 ref 1 00000100:00100000:3.0:1789760735.099793:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713494833949 time 713494573533 state 0x1 ref 1 00000100:00100000:3.0:1789760735.099824:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713494833949 time 713565113052 state 0x0 ref 1 00000100:00100000:3.0:1789760735.099838:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515671936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.099887:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21561344, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.099897:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.099913:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760735.099918:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.099924:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.099937:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.160919:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972581 is committed 00002000:00010000:3.0:1789760735.161140:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21561344 -> 21565440 00010000:00010000:3.0:1789760735.161157:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.161170:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.161180:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.161187:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.161193:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.161201:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.161220:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.161228:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e25a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.161334:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515671936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61494us (62107us total) trans 4294972581 rc 0/0 00000100:00100000:3.0:1789760735.161344:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5293 00000100:00100000:3.0:1789760735.161354:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713494833949 time 713565113052 state 0x2 ref 0 00000100:00100000:2.0:1789760735.166456:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760735.166823:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672064 00000100:00100000:1.0:1789760735.166835:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672064 00000100:00100000:1.0:1789760735.166892:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713494833949 time 713565113052 state 0x0 ref 1 00000100:00100000:1.0:1789760735.166907:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713565373468 time 713565113052 state 0x1 ref 1 00000100:00100000:1.0:1789760735.166972:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713565373468 time 713632260933 state 0x0 ref 1 00000100:00100000:1.0:1789760735.166985:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515672064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760735.167043:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21565440, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:1.0:1789760735.167051:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.167081:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760735.167085:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.167091:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.167104:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.259205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972582 is committed 00002000:00010000:3.0:1789760735.260929:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21565440 -> 21569536 00010000:00010000:3.0:1789760735.261254:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.261270:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.261280:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.261292:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.261298:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.261307:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.261329:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.261336:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e261 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.261438:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515672064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 94450us (94987us total) trans 4294972582 rc 0/0 00000100:00100000:3.0:1789760735.261449:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5294 00000100:00100000:3.0:1789760735.261458:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713565373468 time 713632260933 state 0x2 ref 0 00000100:00100000:0.0:1789760735.267064:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.267149:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672192 00000100:00100000:3.0:1789760735.267158:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672192 00000100:00100000:3.0:1789760735.267210:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713565373468 time 713632260933 state 0x0 ref 1 00000100:00100000:3.0:1789760735.267224:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713632521349 time 713632260933 state 0x1 ref 1 00000100:00100000:3.0:1789760735.267256:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713632521349 time 713732544987 state 0x0 ref 1 00000100:00100000:3.0:1789760735.267271:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515672192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.267318:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21569536, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.267326:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.267342:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760735.267346:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.267352:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.267366:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.299173:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972583 is committed 00002000:00010000:3.0:1789760735.299288:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21569536 -> 21573632 00010000:00010000:3.0:1789760735.299303:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.299316:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.299325:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.299332:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.299336:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.299346:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.299366:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.299373:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e268 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.299459:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515672192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32185us (32401us total) trans 4294972583 rc 0/0 00000100:00100000:3.0:1789760735.299468:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5295 00000100:00100000:3.0:1789760735.299477:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713632521349 time 713732544987 state 0x2 ref 0 00000100:00100000:1.0:1789760735.303450:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760735.303576:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672320 00000100:00100000:0.0:1789760735.303586:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672320 00000100:00100000:0.0:1789760735.303639:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713632521349 time 713732544987 state 0x0 ref 1 00000100:00100000:0.0:1789760735.303654:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713732805403 time 713732544987 state 0x1 ref 1 00000100:00100000:0.0:1789760735.303748:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713732805403 time 713769036655 state 0x0 ref 1 00000100:00100000:0.0:1789760735.303761:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515672320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760735.303806:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21573632, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:0.0:1789760735.303814:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.303830:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760735.303834:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.303841:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.303860:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.337610:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972584 is committed 00002000:00010000:0.0:1789760735.337808:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21573632 -> 21577728 00010000:00010000:0.0:1789760735.337824:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.337838:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.337846:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.337853:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760735.337858:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.337867:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.337886:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.337894:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e26f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.337996:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515672320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34233us (34550us total) trans 4294972584 rc 0/0 00000100:00100000:0.0:1789760735.338005:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5296 00000100:00100000:0.0:1789760735.338014:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713732805403 time 713769036655 state 0x2 ref 0 00000100:00100000:0.0:1789760735.341750:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.341858:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672448 00000100:00100000:3.0:1789760735.341868:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672448 00000100:00100000:3.0:1789760735.341918:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713732805403 time 713769036655 state 0x0 ref 1 00000100:00100000:3.0:1789760735.341934:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713769297071 time 713769036655 state 0x1 ref 1 00000100:00100000:3.0:1789760735.341976:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713769297071 time 713807264108 state 0x0 ref 1 00000100:00100000:3.0:1789760735.341988:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515672448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.342044:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21577728, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.342053:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.342070:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760735.342075:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.342091:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.342106:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.377491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972585 is committed 00002000:00010000:3.0:1789760735.377574:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21577728 -> 21581824 00010000:00010000:3.0:1789760735.377589:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.377602:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.377611:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.377618:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.377623:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.377632:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.377651:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.377659:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e276 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.377856:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515672448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35866us (36144us total) trans 4294972585 rc 0/0 00000100:00100000:3.0:1789760735.377867:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5297 00000100:00100000:3.0:1789760735.377877:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713769297071 time 713807264108 state 0x2 ref 0 00000100:00100000:1.0:1789760735.385808:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760735.385893:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672576 00000100:00100000:2.0:1789760735.385902:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672576 00000100:00100000:2.0:1789760735.386062:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713769297071 time 713807264108 state 0x0 ref 1 00000100:00100000:2.0:1789760735.386093:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713807524524 time 713807264108 state 0x1 ref 1 00000100:00100000:2.0:1789760735.386164:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713807524524 time 713851452042 state 0x0 ref 1 00000100:00100000:2.0:1789760735.386177:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515672576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760735.386225:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21581824, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:2.0:1789760735.386233:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.386249:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760735.386254:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.386260:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.386273:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.434742:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972586 is committed 00002000:00010000:3.0:1789760735.434887:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21581824 -> 21585920 00010000:00010000:3.0:1789760735.434903:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.434917:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.434927:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.434934:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.434940:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.435295:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.435321:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.435328:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e27d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.435514:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515672576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49334us (49712us total) trans 4294972586 rc 0/0 00000100:00100000:3.0:1789760735.435525:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5298 00000100:00100000:3.0:1789760735.435533:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713807524524 time 713851452042 state 0x2 ref 0 00000100:00100000:0.0:1789760735.439445:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.439571:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672704 00000100:00100000:3.0:1789760735.439582:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672704 00000100:00100000:3.0:1789760735.439634:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713807524524 time 713851452042 state 0x0 ref 1 00000100:00100000:3.0:1789760735.439648:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713851712458 time 713851452042 state 0x1 ref 1 00000100:00100000:3.0:1789760735.439684:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713851712458 time 713904972805 state 0x0 ref 1 00000100:00100000:3.0:1789760735.439697:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515672704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.439777:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21585920, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.439786:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.439802:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760735.439807:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.439813:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.439827:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.480698:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972587 is committed 00002000:00010000:3.0:1789760735.481011:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21585920 -> 21590016 00010000:00010000:3.0:1789760735.481035:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.481049:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.481057:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.481064:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.481068:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.481077:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.481098:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.481110:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e284 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.481234:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515672704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41534us (41793us total) trans 4294972587 rc 0/0 00000100:00100000:3.0:1789760735.481244:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5299 00000100:00100000:3.0:1789760735.481255:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713851712458 time 713904972805 state 0x2 ref 0 00000100:00100000:0.0:1789760735.484397:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760735.484485:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672832 00000100:00100000:1.0:1789760735.484495:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672832 00000100:00100000:1.0:1789760735.484548:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713851712458 time 713904972805 state 0x0 ref 1 00000100:00100000:1.0:1789760735.484563:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713905233221 time 713904972805 state 0x1 ref 1 00000100:00100000:1.0:1789760735.484622:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713905233221 time 713949910698 state 0x0 ref 1 00000100:00100000:1.0:1789760735.484635:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515672832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760735.484690:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21590016, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:1.0:1789760735.484699:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.484741:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760735.484745:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.484751:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.484764:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.513908:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972588 is committed 00002000:00010000:1.0:1789760735.514136:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21590016 -> 21594112 00010000:00010000:1.0:1789760735.514151:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.514165:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.514173:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.514181:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760735.514186:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.514195:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.514219:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.514228:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e28b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.514319:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515672832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29681us (29991us total) trans 4294972588 rc 0/0 00000100:00100000:1.0:1789760735.516011:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5300 00000100:00100000:1.0:1789760735.516058:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713905233221 time 713949910698 state 0x2 ref 0 00000100:00100000:1.0:1789760735.519255:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.519497:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515672960 00000100:00100000:3.0:1789760735.519507:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515672960 00000100:00100000:3.0:1789760735.519557:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713905233221 time 713949910698 state 0x0 ref 1 00000100:00100000:3.0:1789760735.519573:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713950171114 time 713949910698 state 0x1 ref 1 00000100:00100000:3.0:1789760735.519603:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713950171114 time 713984892740 state 0x0 ref 1 00000100:00100000:3.0:1789760735.519615:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515672960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.519660:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21594112, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.519668:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.519683:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760735.519687:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.519694:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.519741:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.549871:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760735.550402:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21594112 -> 21598208 00010000:00010000:3.0:1789760735.550418:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.550432:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.550442:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.550449:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.550454:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.550463:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.550481:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.550489:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e292 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.550580:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515672960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30962us (31328us total) trans 4294972589 rc 0/0 00000100:00100000:3.0:1789760735.550590:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5301 00000100:00100000:3.0:1789760735.550600:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713950171114 time 713984892740 state 0x2 ref 0 00000001:00080000:0.0:1789760735.551183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972589 is committed 00000100:00100000:0.0:1789760735.563223:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760735.568849:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673088 00000100:00100000:1.0:1789760735.568865:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673088 00000100:00100000:1.0:1789760735.568930:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713950171114 time 713984892740 state 0x0 ref 1 00000100:00100000:1.0:1789760735.568945:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713985153156 time 713984892740 state 0x1 ref 1 00000100:00100000:3.0:1789760735.569038:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713985153156 time 714034314748 state 0x0 ref 1 00000100:00100000:3.0:1789760735.569134:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515673088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.569184:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21598208, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.569194:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.569209:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.569213:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.569219:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.569234:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.620164:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972590 is committed 00002000:00010000:2.0:1789760735.620276:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21598208 -> 21602304 00010000:00010000:2.0:1789760735.620291:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.620304:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.620314:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.620322:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760735.620327:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.620412:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.620432:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.620440:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e299 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.623768:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515673088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54533us (60449us total) trans 4294972590 rc 0/0 00000100:00100000:2.0:1789760735.623782:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5302 00000100:00100000:2.0:1789760735.623792:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713985153156 time 714034314748 state 0x2 ref 0 00000100:00100000:1.0:1789760735.626829:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.626907:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673216 00000100:00100000:3.0:1789760735.626918:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673216 00000100:00100000:3.0:1789760735.626980:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 713985153156 time 714034314748 state 0x0 ref 1 00000100:00100000:3.0:1789760735.626996:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714034575164 time 714034314748 state 0x1 ref 1 00000100:00100000:3.0:1789760735.627033:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714034575164 time 714092314167 state 0x0 ref 1 00000100:00100000:3.0:1789760735.627045:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515673216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.627093:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21602304, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.627103:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.627118:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.627123:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.627129:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.627142:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.656110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972591 is committed 00002000:00010000:3.0:1789760735.656275:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21602304 -> 21606400 00010000:00010000:3.0:1789760735.656290:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.656302:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.656311:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.656318:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.656323:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.656400:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.656420:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.656427:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.656547:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515673216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29499us (29721us total) trans 4294972591 rc 0/0 00000100:00100000:3.0:1789760735.656557:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5303 00000100:00100000:3.0:1789760735.656566:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714034575164 time 714092314167 state 0x2 ref 0 00000100:00100000:1.0:1789760735.661522:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760735.661680:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673344 00000100:00100000:2.0:1789760735.661690:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673344 00000100:00100000:2.0:1789760735.661771:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714034575164 time 714092314167 state 0x0 ref 1 00000100:00100000:2.0:1789760735.661786:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714092574583 time 714092314167 state 0x1 ref 1 00000100:00100000:2.0:1789760735.661810:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714092574583 time 714127099289 state 0x0 ref 1 00000100:00100000:2.0:1789760735.661820:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515673344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760735.661865:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21606400, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:2.0:1789760735.661874:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.661890:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.661894:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.661900:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.661914:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.699021:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972592 is committed 00002000:00010000:2.0:1789760735.699163:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21606400 -> 21610496 00010000:00010000:2.0:1789760735.699178:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.699189:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.699197:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.699204:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760735.699208:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.699217:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.699235:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.699243:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e2a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.699436:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515673344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37610us (37917us total) trans 4294972592 rc 0/0 00000100:00100000:2.0:1789760735.699447:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5304 00000100:00100000:2.0:1789760735.699458:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714092574583 time 714127099289 state 0x2 ref 0 00000100:00100000:0.0:1789760735.704098:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760735.704200:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673472 00000100:00100000:2.0:1789760735.704211:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673472 00000100:00100000:2.0:1789760735.704262:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714092574583 time 714127099289 state 0x0 ref 1 00000100:00100000:2.0:1789760735.704277:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714127359705 time 714127099289 state 0x1 ref 1 00000100:00100000:2.0:1789760735.704403:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714127359705 time 714169691054 state 0x0 ref 1 00000100:00100000:2.0:1789760735.704416:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515673472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760735.704464:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21610496, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:2.0:1789760735.704473:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.704488:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.704492:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.704498:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.704517:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760735.744548:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21610496 -> 21614592 00010000:00010000:2.0:1789760735.744567:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.744580:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.744590:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.744597:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760735.744603:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.744612:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.744630:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.744639:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e2ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.744769:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515673472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40351us (40678us total) trans 4294972593 rc 0/0 00000100:00100000:2.0:1789760735.744779:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5305 00000100:00100000:2.0:1789760735.744788:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714127359705 time 714169691054 state 0x2 ref 0 00000001:00080000:1.0:1789760735.746584:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972593 is committed 00000100:00100000:1.0:1789760735.749054:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760735.749112:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673600 00000100:00100000:2.0:1789760735.749122:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673600 00000100:00100000:2.0:1789760735.749179:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714127359705 time 714169691054 state 0x0 ref 1 00000100:00100000:2.0:1789760735.749194:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714169951470 time 714169691054 state 0x1 ref 1 00000100:00100000:2.0:1789760735.749220:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714169951470 time 714214508643 state 0x0 ref 1 00000100:00100000:2.0:1789760735.749240:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515673600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760735.749286:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21614592, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:2.0:1789760735.749295:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.749311:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.749315:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.749321:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.749345:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.790209:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21614592 -> 21618688 00010000:00010000:1.0:1789760735.790231:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.790245:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.790253:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.790261:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760735.790265:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.790274:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.790294:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.790302:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e2b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.790399:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515673600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41157us (41348us total) trans 4294972594 rc 0/0 00000100:00100000:1.0:1789760735.790409:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5306 00000100:00100000:1.0:1789760735.790418:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714169951470 time 714214508643 state 0x2 ref 0 00000001:00080000:3.0:1789760735.791628:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972594 is committed 00000100:00100000:2.0:1789760735.795894:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.795991:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673728 00000100:00100000:3.0:1789760735.796000:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673728 00000100:00100000:3.0:1789760735.796063:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714169951470 time 714214508643 state 0x0 ref 1 00000100:00100000:3.0:1789760735.796079:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714214769059 time 714214508643 state 0x1 ref 1 00000100:00100000:3.0:1789760735.796136:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714214769059 time 714261424388 state 0x0 ref 1 00000100:00100000:3.0:1789760735.796148:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515673728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.796198:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21618688, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:3.0:1789760735.796206:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.796222:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.796227:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.796233:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.796246:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.833130:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972595 is committed 00002000:00010000:3.0:1789760735.833211:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21618688 -> 21622784 00010000:00010000:3.0:1789760735.833226:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.833239:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.833248:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.833256:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.833261:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.833271:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.833290:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.833298:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.833498:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515673728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37347us (37610us total) trans 4294972595 rc 0/0 00000100:00100000:3.0:1789760735.833508:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5307 00000100:00100000:3.0:1789760735.833518:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714214769059 time 714261424388 state 0x2 ref 0 00000100:00100000:2.0:1789760735.837124:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.837236:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673856 00000100:00100000:3.0:1789760735.837246:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673856 00000100:00100000:3.0:1789760735.837292:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714214769059 time 714261424388 state 0x0 ref 1 00000100:00100000:3.0:1789760735.837307:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714261684804 time 714261424388 state 0x1 ref 1 00000100:00100000:2.0:1789760735.837414:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714261684804 time 714302700509 state 0x0 ref 1 00000100:00100000:2.0:1789760735.837428:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515673856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760735.837477:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21622784, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa807 00010000:00010000:2.0:1789760735.837486:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.837500:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.837504:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.837510:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.837536:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.872796:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972596 is committed 00002000:00010000:3.0:1789760735.872885:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21622784 -> 21626880 00002000:00010000:3.0:1789760735.872889:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760735 -> 1789760736 00002000:00010000:3.0:1789760735.872892:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760735 -> 1789760736 00010000:00010000:3.0:1789760735.872905:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.872918:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.873053:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.873061:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.873066:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.873075:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.873093:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.873100:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e2c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.873191:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515673856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35761us (36071us total) trans 4294972596 rc 0/0 00000100:00100000:3.0:1789760735.873200:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5308 00000100:00100000:3.0:1789760735.873210:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714261684804 time 714302700509 state 0x2 ref 0 00000100:00100000:2.0:1789760735.877120:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.877244:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515673984 00000100:00100000:3.0:1789760735.877254:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515673984 00000100:00100000:3.0:1789760735.877303:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714261684804 time 714302700509 state 0x0 ref 1 00000100:00100000:3.0:1789760735.877320:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714302960925 time 714302700509 state 0x1 ref 1 00000100:00100000:3.0:1789760735.877456:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714302960925 time 714342744169 state 0x0 ref 1 00000100:00100000:3.0:1789760735.877468:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515673984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.877516:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21626880, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760735.877525:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.877540:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.877544:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.877551:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.877564:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.901447:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21626880 -> 21630976 00010000:00010000:3.0:1789760735.901477:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.901491:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.901502:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.901510:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.901515:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.901524:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.901543:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.901552:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.901642:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515673984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24170us (24526us total) trans 4294972597 rc 0/0 00000100:00100000:3.0:1789760735.901653:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5309 00000100:00100000:3.0:1789760735.901662:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714302960925 time 714342744169 state 0x2 ref 0 00000001:00080000:1.0:1789760735.902242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972597 is committed 00000100:00100000:2.0:1789760735.906177:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.906257:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674112 00000100:00100000:3.0:1789760735.906267:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674112 00000100:00100000:3.0:1789760735.906323:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714302960925 time 714342744169 state 0x0 ref 1 00000100:00100000:3.0:1789760735.906416:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714343004585 time 714342744169 state 0x1 ref 1 00000100:00100000:3.0:1789760735.906446:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714343004585 time 714371736033 state 0x0 ref 1 00000100:00100000:3.0:1789760735.906459:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515674112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.906504:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21630976, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760735.906513:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.906528:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.906532:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.906538:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.906551:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.936260:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21630976 -> 21635072 00010000:00010000:3.0:1789760735.936282:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.936295:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.936304:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.936311:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.936316:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.936324:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.936348:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.936355:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e2d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.936444:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515674112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29983us (30271us total) trans 4294972598 rc 0/0 00000100:00100000:3.0:1789760735.936454:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5310 00000100:00100000:3.0:1789760735.936463:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714343004585 time 714371736033 state 0x2 ref 0 00000001:00080000:1.0:1789760735.936552:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972598 is committed 00000100:00100000:0.0:1789760735.944244:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760735.944345:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674240 00000100:00100000:3.0:1789760735.944354:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674240 00000100:00100000:3.0:1789760735.944409:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714343004585 time 714371736033 state 0x0 ref 1 00000100:00100000:3.0:1789760735.944424:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714371996449 time 714371736033 state 0x1 ref 1 00000100:00100000:3.0:1789760735.944504:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714371996449 time 714409792436 state 0x0 ref 1 00000100:00100000:3.0:1789760735.944518:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515674240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760735.944568:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21635072, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760735.944578:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.944595:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.944600:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.944607:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.944622:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.978769:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972599 is committed 00002000:00010000:3.0:1789760735.978933:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21635072 -> 21639168 00010000:00010000:3.0:1789760735.979019:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.979100:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.979111:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.979119:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760735.979124:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.979134:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.979156:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.979169:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.979303:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515674240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34783us (35063us total) trans 4294972599 rc 0/0 00000100:00100000:3.0:1789760735.979314:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5311 00000100:00100000:3.0:1789760735.979323:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714371996449 time 714409792436 state 0x2 ref 0 00000100:00100000:3.0:1789760735.983728:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760735.983854:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674368 00000100:00100000:1.0:1789760735.983864:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674368 00000100:00100000:1.0:1789760735.983912:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714371996449 time 714409792436 state 0x0 ref 1 00000100:00100000:1.0:1789760735.983927:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714410052852 time 714409792436 state 0x1 ref 1 00000100:00100000:1.0:1789760735.983966:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714410052852 time 714449243404 state 0x0 ref 1 00000100:00100000:1.0:1789760735.983979:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515674368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760735.984046:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21639168, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:1.0:1789760735.984055:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.984070:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.984074:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.984080:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.984094:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.025948:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972600 is committed 00002000:00010000:3.0:1789760736.026126:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21639168 -> 21643264 00010000:00010000:3.0:1789760736.026143:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.026158:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.026167:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.026175:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.026180:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.026189:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.026215:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.026222:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e2df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.026389:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515674368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42407us (42681us total) trans 4294972600 rc 0/0 00000100:00100000:3.0:1789760736.026399:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5312 00000100:00100000:3.0:1789760736.026408:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714410052852 time 714449243404 state 0x2 ref 0 00000100:00100000:1.0:1789760736.033488:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760736.033580:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674496 00000100:00100000:2.0:1789760736.033590:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674496 00000100:00100000:2.0:1789760736.033665:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714410052852 time 714449243404 state 0x0 ref 1 00000100:00100000:2.0:1789760736.033681:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714449503820 time 714449243404 state 0x1 ref 1 00000100:00100000:2.0:1789760736.033744:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714449503820 time 714498997696 state 0x0 ref 1 00000100:00080000:1.0:1789760736.033886:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760736.033922:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515674496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760736.033984:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21643264, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:2.0:1789760736.033993:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.034009:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760736.034014:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.034020:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.034047:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760736.069379:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21643264 -> 21647360 00010000:00010000:2.0:1789760736.069397:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.069410:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.069419:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.069427:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760736.069433:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.069442:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.069460:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.069467:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e2e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.069553:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515674496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35794us (36071us total) trans 4294972601 rc 0/0 00000100:00100000:2.0:1789760736.069563:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5313 00000100:00100000:2.0:1789760736.069573:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714449503820 time 714498997696 state 0x2 ref 0 00000001:00080000:2.0:1789760736.069939:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972601 is committed 00000100:00100000:1.0:1789760736.073506:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760736.073591:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674624 00000100:00100000:2.0:1789760736.073601:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674624 00000100:00100000:2.0:1789760736.073650:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714449503820 time 714498997696 state 0x0 ref 1 00000100:00100000:2.0:1789760736.073664:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714499258112 time 714498997696 state 0x1 ref 1 00000100:00100000:2.0:1789760736.073865:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714499258112 time 714539152363 state 0x0 ref 1 00000100:00100000:2.0:1789760736.073878:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515674624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760736.073927:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21647360, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:2.0:1789760736.073935:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.074047:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760736.074052:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.074059:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.074073:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.102253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972602 is committed 00002000:00010000:2.0:1789760736.102363:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21647360 -> 21651456 00010000:00010000:2.0:1789760736.102378:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.102391:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.102399:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.102406:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760736.102411:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.102421:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.102441:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.102449:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e2ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.102545:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515674624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28657us (29034us total) trans 4294972602 rc 0/0 00000100:00100000:2.0:1789760736.102556:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5314 00000100:00100000:2.0:1789760736.102566:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714499258112 time 714539152363 state 0x2 ref 0 00000100:00100000:3.0:1789760736.112129:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760736.112643:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674752 00000100:00100000:1.0:1789760736.112652:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674752 00000100:00100000:1.0:1789760736.112742:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714499258112 time 714539152363 state 0x0 ref 1 00000100:00100000:1.0:1789760736.112759:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714539412779 time 714539152363 state 0x1 ref 1 00000100:00100000:1.0:1789760736.112786:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714539412779 time 714578075417 state 0x0 ref 1 00000100:00100000:1.0:1789760736.112798:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515674752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760736.112845:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21651456, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:1.0:1789760736.112855:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.112871:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760736.112875:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.112882:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.112897:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.177438:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972603 is committed 00002000:00010000:1.0:1789760736.177614:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21651456 -> 21655552 00010000:00010000:1.0:1789760736.177630:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.177642:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.177652:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.177660:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760736.177665:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.177674:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.177692:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.177701:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e2f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.177813:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515674752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65011us (65687us total) trans 4294972603 rc 0/0 00000100:00100000:1.0:1789760736.177823:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5315 00000100:00100000:1.0:1789760736.177833:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714539412779 time 714578075417 state 0x2 ref 0 00000100:00100000:1.0:1789760736.185254:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.185346:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515674880 00000100:00100000:3.0:1789760736.185355:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515674880 00000100:00100000:3.0:1789760736.185405:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714539412779 time 714578075417 state 0x0 ref 1 00000100:00100000:3.0:1789760736.185420:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714578335833 time 714578075417 state 0x1 ref 1 00000100:00100000:3.0:1789760736.185449:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714578335833 time 714650738658 state 0x0 ref 1 00000100:00100000:3.0:1789760736.185462:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515674880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.185506:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21655552, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.185515:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.185537:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760736.185541:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.185547:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.185561:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760736.231254:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21655552 -> 21659648 00010000:00010000:1.0:1789760736.231274:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.231292:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.231301:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.231309:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760736.231313:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.231323:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.231343:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.231350:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e2fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.231441:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515674880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45976us (46191us total) trans 4294972604 rc 0/0 00000100:00100000:1.0:1789760736.231452:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5316 00000100:00100000:1.0:1789760736.231471:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714578335833 time 714650738658 state 0x2 ref 0 00000001:00080000:2.0:1789760736.232216:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972604 is committed 00000100:00100000:2.0:1789760736.234671:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.234852:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675008 00000100:00100000:3.0:1789760736.234863:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675008 00000100:00100000:3.0:1789760736.234909:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714578335833 time 714650738658 state 0x0 ref 1 00000100:00100000:3.0:1789760736.234926:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714650999074 time 714650738658 state 0x1 ref 1 00000100:00100000:3.0:1789760736.234954:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714650999074 time 714700242798 state 0x0 ref 1 00000100:00100000:3.0:1789760736.234966:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.235012:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21659648, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.235040:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.235055:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760736.235060:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.235067:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.235288:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.293333:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21659648 -> 21663744 00010000:00010000:3.0:1789760736.293354:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.293367:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.293375:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.293383:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.293388:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.293397:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.293416:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.293424:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e302 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.293539:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515675008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58569us (58871us total) trans 4294972605 rc 0/0 00000100:00100000:3.0:1789760736.293550:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5317 00000100:00100000:3.0:1789760736.293559:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714650999074 time 714700242798 state 0x2 ref 0 00000001:00080000:1.0:1789760736.294191:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972605 is committed 00000100:00100000:3.0:1789760736.298070:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760736.298225:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675136 00000100:00100000:1.0:1789760736.298235:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675136 00000100:00100000:1.0:1789760736.298287:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714650999074 time 714700242798 state 0x0 ref 1 00000100:00100000:1.0:1789760736.298302:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714700503214 time 714700242798 state 0x1 ref 1 00000100:00100000:1.0:1789760736.298329:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714700503214 time 714763618405 state 0x0 ref 1 00000100:00100000:1.0:1789760736.298340:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760736.298387:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21663744, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:1.0:1789760736.298396:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.298412:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760736.298416:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.298422:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.298434:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.340884:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21663744 -> 21667840 00010000:00010000:3.0:1789760736.340907:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.340920:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.340930:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.340938:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.340943:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.340951:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.340970:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.340984:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e309 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.341278:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515675136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42741us (43019us total) trans 4294972606 rc 0/0 00000100:00100000:3.0:1789760736.341291:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5318 00000100:00100000:3.0:1789760736.341300:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714700503214 time 714763618405 state 0x2 ref 0 00000001:00080000:2.0:1789760736.341703:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972606 is committed 00000100:00100000:2.0:1789760736.351639:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.351826:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675264 00000100:00100000:3.0:1789760736.351837:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675264 00000100:00100000:3.0:1789760736.351890:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714700503214 time 714763618405 state 0x0 ref 1 00000100:00100000:3.0:1789760736.351906:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714763878821 time 714763618405 state 0x1 ref 1 00000100:00100000:3.0:1789760736.351939:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714763878821 time 714817228154 state 0x0 ref 1 00000100:00100000:3.0:1789760736.351952:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515675264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.351999:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21667840, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.352007:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.352034:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760736.352039:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.352045:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.352064:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.391936:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972607 is committed 00002000:00010000:3.0:1789760736.392653:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21667840 -> 21671936 00010000:00010000:3.0:1789760736.392852:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.392867:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.392878:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.392886:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.392891:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.392900:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.392920:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.392928:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e310 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.393140:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515675264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41184us (41506us total) trans 4294972607 rc 0/0 00000100:00100000:3.0:1789760736.393151:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5319 00000100:00100000:3.0:1789760736.393161:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714763878821 time 714817228154 state 0x2 ref 0 00000100:00100000:1.0:1789760736.397474:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760736.397590:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675392 00000100:00100000:2.0:1789760736.397601:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675392 00000100:00100000:2.0:1789760736.397655:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714763878821 time 714817228154 state 0x0 ref 1 00000100:00100000:2.0:1789760736.397901:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714817488570 time 714817228154 state 0x1 ref 1 00000100:00100000:2.0:1789760736.398018:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714817488570 time 714863306100 state 0x0 ref 1 00000100:00100000:2.0:1789760736.398041:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760736.398092:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21671936, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:2.0:1789760736.398102:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.398119:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760736.398124:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.398131:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.398145:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760736.430164:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21671936 -> 21676032 00010000:00010000:2.0:1789760736.430182:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.430219:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.430230:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.430238:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760736.430243:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.430253:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.430273:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.430281:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e317 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.430370:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515675392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32327us (32900us total) trans 4294972608 rc 0/0 00000100:00100000:2.0:1789760736.430380:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5320 00000100:00100000:2.0:1789760736.430389:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714817488570 time 714863306100 state 0x2 ref 0 00000001:00080000:2.0:1789760736.430425:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972608 is committed 00000100:00100000:2.0:1789760736.435831:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.436110:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675520 00000100:00100000:3.0:1789760736.436119:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675520 00000100:00100000:3.0:1789760736.436173:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714817488570 time 714863306100 state 0x0 ref 1 00000100:00100000:3.0:1789760736.436189:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714863566516 time 714863306100 state 0x1 ref 1 00000100:00100000:3.0:1789760736.436253:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714863566516 time 714901541662 state 0x0 ref 1 00000100:00100000:3.0:1789760736.436266:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515675520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.436315:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21676032, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.436324:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.436340:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760736.436344:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.436350:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.436364:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.471309:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972609 is committed 00002000:00010000:3.0:1789760736.471400:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21676032 -> 21680128 00010000:00010000:3.0:1789760736.471416:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.471428:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.471438:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.471446:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.471450:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.471459:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.471478:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.471486:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e31e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.471608:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515675520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35338us (35781us total) trans 4294972609 rc 0/0 00000100:00100000:3.0:1789760736.471623:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5321 00000100:00100000:3.0:1789760736.471632:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714863566516 time 714901541662 state 0x2 ref 0 00000100:00100000:2.0:1789760736.475390:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760736.475495:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675648 00000100:00100000:1.0:1789760736.475504:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675648 00000100:00100000:1.0:1789760736.475553:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714863566516 time 714901541662 state 0x0 ref 1 00000100:00100000:1.0:1789760736.475568:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714901802078 time 714901541662 state 0x1 ref 1 00000100:00100000:1.0:1789760736.475629:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714901802078 time 714940918136 state 0x0 ref 1 00000100:00100000:1.0:1789760736.475642:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760736.475815:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21680128, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:1.0:1789760736.475824:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.475840:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760736.475845:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.475851:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.475865:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.508626:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972610 is committed 00002000:00010000:0.0:1789760736.508840:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21680128 -> 21684224 00010000:00010000:0.0:1789760736.508857:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.508870:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.508879:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.508887:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760736.508893:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.508902:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.508921:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.508928:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e325 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.509037:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33392us (33652us total) trans 4294972610 rc 0/0 00000100:00100000:0.0:1789760736.509047:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5322 00000100:00100000:0.0:1789760736.509057:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714901802078 time 714940918136 state 0x2 ref 0 00000100:00100000:1.0:1789760736.512564:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.512669:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675776 00000100:00100000:3.0:1789760736.512679:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675776 00000100:00100000:3.0:1789760736.512882:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714901802078 time 714940918136 state 0x0 ref 1 00000100:00100000:3.0:1789760736.512898:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714941178552 time 714940918136 state 0x1 ref 1 00000100:00100000:3.0:1789760736.512971:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714941178552 time 714978258418 state 0x0 ref 1 00000100:00100000:3.0:1789760736.512984:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515675776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.513039:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21684224, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.513047:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.513061:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760736.513065:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.513071:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.513085:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.536473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972611 is committed 00002000:00010000:3.0:1789760736.536538:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21684224 -> 21688320 00010000:00010000:3.0:1789760736.536553:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.536566:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.536576:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.536584:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.536589:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.536598:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.536619:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.536626:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e32c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.536774:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515675776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23788us (24215us total) trans 4294972611 rc 0/0 00000100:00100000:3.0:1789760736.536785:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5323 00000100:00100000:3.0:1789760736.536795:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714941178552 time 714978258418 state 0x2 ref 0 00000100:00100000:2.0:1789760736.540093:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760736.540297:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515675904 00000100:00100000:1.0:1789760736.540307:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515675904 00000100:00100000:1.0:1789760736.540359:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714941178552 time 714978258418 state 0x0 ref 1 00000100:00100000:1.0:1789760736.540374:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714978518834 time 714978258418 state 0x1 ref 1 00000100:00100000:1.0:1789760736.540434:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714978518834 time 715005722253 state 0x0 ref 1 00000100:00100000:1.0:1789760736.540447:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760736.540496:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21688320, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:1.0:1789760736.540505:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.540521:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760736.540526:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.540532:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.540545:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.565422:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972612 is committed 00002000:00010000:2.0:1789760736.565583:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21688320 -> 21692416 00010000:00010000:2.0:1789760736.565598:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.565608:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.565614:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.565619:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760736.565623:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.565629:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.565643:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.565647:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e333 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.565790:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515675904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25298us (25657us total) trans 4294972612 rc 0/0 00000100:00100000:2.0:1789760736.565800:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5324 00000100:00100000:2.0:1789760736.565807:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714978518834 time 715005722253 state 0x2 ref 0 00000100:00100000:2.0:1789760736.570685:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.570870:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676032 00000100:00100000:3.0:1789760736.570880:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676032 00000100:00100000:3.0:1789760736.570930:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 714978518834 time 715005722253 state 0x0 ref 1 00000100:00100000:3.0:1789760736.570945:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715005982669 time 715005722253 state 0x1 ref 1 00000100:00100000:3.0:1789760736.571065:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715005982669 time 715036353732 state 0x0 ref 1 00000100:00100000:3.0:1789760736.571077:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515676032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.571123:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21692416, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.571132:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.571149:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.571153:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.571160:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.571173:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.574817:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760736.591345:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21692416 -> 21696512 00010000:00010000:2.0:1789760736.591363:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591377:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591387:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591395:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760736.591400:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591409:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591427:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591434:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e33a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.591518:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515676032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 20438us (21041us total) trans 4294972613 rc 0/0 00000100:00100000:2.0:1789760736.591528:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5325 00000100:00100000:2.0:1789760736.591538:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715005982669 time 715036353732 state 0x2 ref 0 00000001:00080000:0.0:1789760736.591610:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972613 is committed 00000100:00100000:0.0:1789760736.596194:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.596332:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676160 00000100:00100000:3.0:1789760736.596342:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676160 00000100:00100000:3.0:1789760736.596393:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715005982669 time 715036353732 state 0x0 ref 1 00000100:00100000:3.0:1789760736.596407:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715036614148 time 715036353732 state 0x1 ref 1 00000100:00100000:3.0:1789760736.596470:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715036614148 time 715061758581 state 0x0 ref 1 00000100:00100000:3.0:1789760736.596482:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515676160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.596537:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21696512, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.596546:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.596561:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.596564:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.596571:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.596584:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.624901:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972614 is committed 00002000:00010000:3.0:1789760736.625050:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21696512 -> 21700608 00010000:00010000:3.0:1789760736.625065:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.625142:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.625154:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.625162:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.625167:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.625177:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.625195:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.625202:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e341 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.625324:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515676160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28839us (29136us total) trans 4294972614 rc 0/0 00000100:00100000:3.0:1789760736.625335:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5326 00000100:00100000:3.0:1789760736.625345:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715036614148 time 715061758581 state 0x2 ref 0 00000100:00100000:2.0:1789760736.632873:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.632982:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676288 00000100:00100000:3.0:1789760736.632992:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676288 00000100:00100000:3.0:1789760736.633060:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715036614148 time 715061758581 state 0x0 ref 1 00000100:00100000:3.0:1789760736.633077:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715062018997 time 715061758581 state 0x1 ref 1 00000100:00100000:3.0:1789760736.633141:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715062018997 time 715098429117 state 0x0 ref 1 00000100:00100000:3.0:1789760736.633154:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515676288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.633203:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21700608, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.633212:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.633231:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.633236:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.633242:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.633257:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.667144:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972615 is committed 00002000:00010000:3.0:1789760736.667277:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21700608 -> 21704704 00010000:00010000:3.0:1789760736.667294:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.667308:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.667316:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.667324:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760736.667328:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.667338:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.667361:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.667368:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e348 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.667462:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515676288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34306us (34593us total) trans 4294972615 rc 0/0 00000100:00100000:3.0:1789760736.667473:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5327 00000100:00100000:3.0:1789760736.667483:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715062018997 time 715098429117 state 0x2 ref 0 00000100:00100000:2.0:1789760736.670625:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760736.670751:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676416 00000100:00100000:0.0:1789760736.670762:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676416 00000100:00100000:0.0:1789760736.670817:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715062018997 time 715098429117 state 0x0 ref 1 00000100:00100000:0.0:1789760736.670831:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715098689533 time 715098429117 state 0x1 ref 1 00000100:00100000:0.0:1789760736.670900:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715098689533 time 715136180791 state 0x0 ref 1 00000100:00100000:0.0:1789760736.670919:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760736.670967:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21704704, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:0.0:1789760736.670976:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.670993:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.670998:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.671004:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.671017:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.720456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972616 is committed 00002000:00010000:0.0:1789760736.720558:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21704704 -> 21708800 00010000:00010000:0.0:1789760736.720574:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.720586:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.720596:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.720603:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760736.720608:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.720617:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.720635:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.720643:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e34f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.720856:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49934us (50234us total) trans 4294972616 rc 0/0 00000100:00100000:0.0:1789760736.720868:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5328 00000100:00100000:0.0:1789760736.720877:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715098689533 time 715136180791 state 0x2 ref 0 00000100:00100000:2.0:1789760736.724229:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760736.724310:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676544 00000100:00100000:0.0:1789760736.724319:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676544 00000100:00100000:0.0:1789760736.724371:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715098689533 time 715136180791 state 0x0 ref 1 00000100:00100000:0.0:1789760736.724387:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715136441207 time 715136180791 state 0x1 ref 1 00000100:00100000:0.0:1789760736.724417:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715136441207 time 715189705934 state 0x0 ref 1 00000100:00100000:0.0:1789760736.724430:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760736.724476:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21708800, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:0.0:1789760736.724485:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.724502:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.724506:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.724512:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.724525:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.755265:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21708800 -> 21712896 00010000:00010000:0.0:1789760736.755287:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.755302:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.755310:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.755317:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760736.755323:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.755332:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.755352:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.755360:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e356 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.755459:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515676544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31026us (31234us total) trans 4294972617 rc 0/0 00000100:00100000:0.0:1789760736.755470:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5329 00000100:00100000:0.0:1789760736.755481:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715136441207 time 715189705934 state 0x2 ref 0 00000001:00080000:2.0:1789760736.755627:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972617 is committed 00000100:00100000:2.0:1789760736.759516:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760736.759629:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676672 00000100:00100000:1.0:1789760736.759639:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676672 00000100:00100000:1.0:1789760736.759700:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715136441207 time 715189705934 state 0x0 ref 1 00000100:00100000:1.0:1789760736.759860:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715189966350 time 715189705934 state 0x1 ref 1 00000100:00100000:1.0:1789760736.759892:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715189966350 time 715225180916 state 0x0 ref 1 00000100:00100000:1.0:1789760736.759905:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760736.759959:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21712896, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:1.0:1789760736.759968:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.759986:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760736.759992:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.759999:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.760013:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.766348:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544411264:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.766418:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.766478:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760736.766486:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8010e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544410880:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.766514:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.766539:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544411392:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.766564:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.766590:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8010e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544411008:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.766612:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.766629:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c62a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544411520:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.766649:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.766678:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760736.766684:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c61f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544411136:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.766732:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.766764:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411264 00000100:00100000:2.0:1789760736.766772:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411264 00000100:00100000:2.0:1789760736.767407:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411520 00000100:00100000:2.0:1789760736.767412:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411520 00000100:00100000:0.0:1789760736.767875:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789760736.767888:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544411264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.767965:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544411264:12345-0@lo:400:kworker.0 Request processed in 80us (1546us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.767973:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789760736.767991:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:0.0:1789760736.767999:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544411520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.768046:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544411520:12345-0@lo:400:kworker.0 Request processed in 48us (1395us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.768052:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:2.0:1789760736.768241:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544410880 00000100:00100000:2.0:1789760736.768246:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544410880 00000100:00100000:2.0:1789760736.768275:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789760736.768282:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909351c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544410880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768316:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909351c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544410880:12345-0@lo:400:kworker.0 Request processed in 34us (1800us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.768322:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789760736.768349:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411392 00000100:00100000:2.0:1789760736.768353:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411392 00000100:00100000:2.0:1789760736.768378:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411008 00000100:00100000:2.0:1789760736.768382:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411008 00000100:00100000:2.0:1789760736.768399:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411136 00000100:00100000:2.0:1789760736.768402:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411136 00000100:00100000:2.0:1789760736.768417:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789760736.768422:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544411392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768452:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544411392:12345-0@lo:400:kworker.0 Request processed in 29us (1887us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.768458:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789760736.768470:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789760736.768475:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544411008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768503:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544411008:12345-0@lo:400:kworker.0 Request processed in 27us (1890us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.768509:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789760736.768520:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:2.0:1789760736.768526:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544411136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768554:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544411136:12345-0@lo:400:kworker.0 Request processed in 28us (1822us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.768560:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00080000:2.0:1789760736.768586:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760736.768591:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760736.768593:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760736.768629:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c61f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544411136:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768641:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8010e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544411008:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768651:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c62a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544411520:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768832:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8010e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544410880:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768847:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544411392:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.768858:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544411264:0@lo:400:kworker.0 00000001:00080000:3.0:1789760736.793497:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972618 is committed 00002000:00010000:0.0:1789760736.793634:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21712896 -> 21716992 00010000:00010000:0.0:1789760736.793651:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.793747:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.793761:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.793768:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760736.793772:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.793781:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.793800:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.793807:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e35d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.793893:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33986us (34381us total) trans 4294972618 rc 0/0 00000100:00100000:0.0:1789760736.793902:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5330 00000100:00100000:0.0:1789760736.793912:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715189966350 time 715225180916 state 0x2 ref 0 00000100:00100000:2.0:1789760736.797812:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.797903:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676800 00000100:00100000:3.0:1789760736.797912:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676800 00000100:00100000:3.0:1789760736.798058:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715189966350 time 715225180916 state 0x0 ref 1 00000100:00100000:3.0:1789760736.798073:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715225441332 time 715225180916 state 0x1 ref 1 00000100:00100000:3.0:1789760736.798100:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715225441332 time 715263389573 state 0x0 ref 1 00000100:00100000:3.0:1789760736.798112:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.798156:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21716992, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.798165:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.798185:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.798189:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.798195:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.798208:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.832313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972619 is committed 00002000:00010000:0.0:1789760736.832447:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21716992 -> 21721088 00010000:00010000:0.0:1789760736.832463:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.832482:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.832492:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.832499:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760736.832504:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.832513:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.832533:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.832542:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e364 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.832670:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515676800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34554us (34863us total) trans 4294972619 rc 0/0 00000100:00100000:0.0:1789760736.832682:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5331 00000100:00100000:0.0:1789760736.832691:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715225441332 time 715263389573 state 0x2 ref 0 00000100:00100000:2.0:1789760736.837525:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.837614:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515676928 00000100:00100000:3.0:1789760736.837624:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515676928 00000100:00100000:3.0:1789760736.837677:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715225441332 time 715263389573 state 0x0 ref 1 00000100:00100000:3.0:1789760736.837693:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715263649989 time 715263389573 state 0x1 ref 1 00000100:00100000:3.0:1789760736.837919:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715263649989 time 715303205946 state 0x0 ref 1 00000100:00100000:3.0:1789760736.837933:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515676928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.838123:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21721088, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa807 00010000:00010000:3.0:1789760736.838131:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.838147:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.838152:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.838158:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.838171:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.893095:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21721088 -> 21725184 00002000:00010000:0.0:1789760736.893103:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760736 -> 1789760737 00002000:00010000:0.0:1789760736.893106:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760736 -> 1789760737 00010000:00010000:0.0:1789760736.893117:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.893130:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.893140:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.893148:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760736.893153:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.893162:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.893183:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.893191:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e36b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.893285:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515676928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55349us (55764us total) trans 4294972620 rc 0/0 00000100:00100000:0.0:1789760736.893295:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5332 00000100:00100000:0.0:1789760736.893305:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715263649989 time 715303205946 state 0x2 ref 0 00000001:00080000:1.0:1789760736.894046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972620 is committed 00000100:00100000:2.0:1789760736.900820:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760736.900901:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677056 00000100:00100000:3.0:1789760736.900911:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677056 00000100:00100000:3.0:1789760736.901038:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715263649989 time 715303205946 state 0x0 ref 1 00000100:00100000:3.0:1789760736.901054:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715303466362 time 715303205946 state 0x1 ref 1 00000100:00100000:3.0:1789760736.901126:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715303466362 time 715366415083 state 0x0 ref 1 00000100:00100000:3.0:1789760736.901140:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515677056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760736.901185:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21725184, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:3.0:1789760736.901194:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.901209:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.901213:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.901219:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.901232:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.937740:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972621 is committed 00002000:00010000:1.0:1789760736.937863:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21725184 -> 21729280 00010000:00010000:1.0:1789760736.937878:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.937890:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.937899:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.937906:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760736.937911:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.937920:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.937997:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.938011:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e372 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.938145:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515677056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37002us (37329us total) trans 4294972621 rc 0/0 00000100:00100000:1.0:1789760736.938155:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5333 00000100:00100000:1.0:1789760736.938164:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715303466362 time 715366415083 state 0x2 ref 0 00000100:00100000:2.0:1789760736.948105:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760736.948222:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677184 00000100:00100000:0.0:1789760736.948233:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677184 00000100:00100000:0.0:1789760736.948290:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715303466362 time 715366415083 state 0x0 ref 1 00000100:00100000:0.0:1789760736.948305:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715366675499 time 715366415083 state 0x1 ref 1 00000100:00100000:0.0:1789760736.948464:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715366675499 time 715413751582 state 0x0 ref 1 00000100:00100000:0.0:1789760736.948478:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515677184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760736.948524:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21729280, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:0.0:1789760736.948534:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.948550:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.948555:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.948561:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.948576:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.014553:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972622 is committed 00002000:00010000:1.0:1789760737.014702:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21729280 -> 21733376 00010000:00010000:1.0:1789760737.014738:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.014751:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.014760:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.014767:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760737.014771:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.014779:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.014803:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.014811:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e379 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.014906:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515677184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66426us (66806us total) trans 4294972622 rc 0/0 00000100:00100000:1.0:1789760737.014916:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5334 00000100:00100000:1.0:1789760737.014925:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715366675499 time 715413751582 state 0x2 ref 0 00000100:00100000:2.0:1789760737.021590:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760737.021703:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677312 00000100:00100000:0.0:1789760737.021783:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677312 00000100:00100000:0.0:1789760737.021838:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715366675499 time 715413751582 state 0x0 ref 1 00000100:00100000:0.0:1789760737.021853:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715414011998 time 715413751582 state 0x1 ref 1 00000100:00100000:0.0:1789760737.022042:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715414011998 time 715487329326 state 0x0 ref 1 00000100:00100000:0.0:1789760737.022057:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515677312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760737.022118:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21733376, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:0.0:1789760737.022128:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.022143:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760737.022148:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.022154:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.022166:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.065085:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972623 is committed 00002000:00010000:0.0:1789760737.065240:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21733376 -> 21737472 00010000:00010000:0.0:1789760737.065256:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.065270:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.065279:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.065287:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760737.065292:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.065301:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.065320:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.065328:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e380 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.065422:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515677312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43365us (43837us total) trans 4294972623 rc 0/0 00000100:00100000:0.0:1789760737.065432:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5335 00000100:00100000:0.0:1789760737.065442:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715414011998 time 715487329326 state 0x2 ref 0 00000100:00100000:2.0:1789760737.070414:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760737.070491:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677440 00000100:00100000:0.0:1789760737.070502:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677440 00000100:00100000:0.0:1789760737.070554:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715414011998 time 715487329326 state 0x0 ref 1 00000100:00100000:0.0:1789760737.070569:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715487589742 time 715487329326 state 0x1 ref 1 00000100:00100000:0.0:1789760737.070601:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715487589742 time 715535889894 state 0x0 ref 1 00000100:00100000:0.0:1789760737.070612:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515677440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760737.070660:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21737472, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:0.0:1789760737.070680:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.070696:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760737.070701:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.070743:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.070758:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.125666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972624 is committed 00002000:00010000:1.0:1789760737.125823:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21737472 -> 21741568 00010000:00010000:1.0:1789760737.125840:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125853:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125863:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125872:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760737.125877:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125892:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125910:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.125919:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e387 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.126087:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515677440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55471us (55680us total) trans 4294972624 rc 0/0 00000100:00100000:1.0:1789760737.126098:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5336 00000100:00100000:1.0:1789760737.126108:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715487589742 time 715535889894 state 0x2 ref 0 00000100:00100000:2.0:1789760737.138183:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.141872:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677568 00000100:00100000:3.0:1789760737.141889:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677568 00000100:00100000:3.0:1789760737.142255:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715487589742 time 715535889894 state 0x0 ref 1 00000100:00100000:3.0:1789760737.142272:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715536150310 time 715535889894 state 0x1 ref 1 00000100:00100000:3.0:1789760737.142358:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715536150310 time 715607646124 state 0x0 ref 1 00000100:00100000:3.0:1789760737.142374:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515677568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.142422:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21741568, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:3.0:1789760737.142430:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.142464:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760737.142469:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.142475:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.142488:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.187914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972625 is committed 00002000:00010000:2.0:1789760737.187981:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21741568 -> 21745664 00010000:00010000:2.0:1789760737.187996:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.188010:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.188030:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.188038:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.188042:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.188057:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.188076:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.188084:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e38e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.188172:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515677568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45799us (49993us total) trans 4294972625 rc 0/0 00000100:00100000:2.0:1789760737.188182:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5337 00000100:00100000:2.0:1789760737.188205:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715536150310 time 715607646124 state 0x2 ref 0 00000100:00100000:2.0:1789760737.191331:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.191488:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677696 00000100:00100000:3.0:1789760737.191498:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677696 00000100:00100000:3.0:1789760737.191550:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715536150310 time 715607646124 state 0x0 ref 1 00000100:00100000:3.0:1789760737.191565:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715607906540 time 715607646124 state 0x1 ref 1 00000100:00100000:3.0:1789760737.191600:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715607906540 time 715656888607 state 0x0 ref 1 00000100:00100000:3.0:1789760737.191617:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515677696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.191672:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21745664, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:3.0:1789760737.191681:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.191697:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760737.191702:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.191739:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.191761:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.202166:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760737.202213:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760737.202291:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760737.202318:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677824 00000100:00100000:2.0:1789760737.202326:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677824 00000100:00100000:2.0:1789760737.202370:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 390 00000100:00100000:2.0:1789760737.202380:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909350c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515677824:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760737.202444:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909350c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515677824:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 65us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760737.202451:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 390 00000100:00100000:2.0:1789760737.202482:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515677952 00000100:00100000:2.0:1789760737.202486:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515677952 00000100:00100000:2.0:1789760737.202517:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678080 00000100:00100000:2.0:1789760737.202520:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678080 00000100:00100000:2.0:1789760737.202537:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 451 00000100:00100000:2.0:1789760737.202543:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515677952:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760737.202572:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515677952:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 29us (417us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760737.202579:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 451 00000100:00100000:2.0:1789760737.202589:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 452 00000100:00100000:2.0:1789760737.202595:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2365:x1876699515678080:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760737.202615:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2365:x1876699515678080:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 20us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760737.202621:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 452 00000100:00080000:2.0:1789760737.202639:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760737.246080:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972626 is committed 00002000:00010000:2.0:1789760737.246235:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21745664 -> 21749760 00010000:00010000:2.0:1789760737.246251:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.246262:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.246270:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.246277:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.246281:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.246590:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.246612:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.246621:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e395 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.247418:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515677696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55796us (56088us total) trans 4294972626 rc 0/0 00000100:00100000:2.0:1789760737.247431:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5338 00000100:00100000:2.0:1789760737.247441:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715607906540 time 715656888607 state 0x2 ref 0 00000100:00100000:2.0:1789760737.252984:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760737.253065:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678208 00000100:00100000:1.0:1789760737.253137:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678208 00000100:00100000:1.0:1789760737.253192:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715607906540 time 715656888607 state 0x0 ref 1 00000100:00100000:1.0:1789760737.253278:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715657149023 time 715656888607 state 0x1 ref 1 00000100:00100000:1.0:1789760737.253314:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715657149023 time 715718602774 state 0x0 ref 1 00000100:00100000:1.0:1789760737.253326:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515678208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760737.253436:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21749760, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:1.0:1789760737.253445:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.253771:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760737.253777:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.253784:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.253799:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.306492:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972627 is committed 00002000:00010000:2.0:1789760737.306595:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21749760 -> 21753856 00010000:00010000:2.0:1789760737.306611:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.306624:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.306632:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.306644:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.306648:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.306657:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.306676:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.306685:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e39c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.306811:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515678208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53481us (53830us total) trans 4294972627 rc 0/0 00000100:00100000:2.0:1789760737.306825:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5339 00000100:00100000:2.0:1789760737.306835:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715657149023 time 715718602774 state 0x2 ref 0 00000100:00100000:3.0:1789760737.319095:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760737.321024:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678336 00000100:00100000:1.0:1789760737.321045:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678336 00000100:00100000:1.0:1789760737.321112:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715657149023 time 715718602774 state 0x0 ref 1 00000100:00100000:1.0:1789760737.321127:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715718863190 time 715718602774 state 0x1 ref 1 00000100:00100000:1.0:1789760737.321157:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715718863190 time 715786446073 state 0x0 ref 1 00000100:00100000:1.0:1789760737.321169:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515678336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760737.321298:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21753856, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:1.0:1789760737.321307:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.321334:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760737.321338:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.321357:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.321391:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.379205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972628 is committed 00002000:00010000:2.0:1789760737.379304:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21753856 -> 21757952 00010000:00010000:2.0:1789760737.379319:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.379332:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.379346:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.379353:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.379358:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.379367:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.379387:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.379405:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e3a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.379494:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515678336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58323us (60404us total) trans 4294972628 rc 0/0 00000100:00100000:2.0:1789760737.379504:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5340 00000100:00100000:2.0:1789760737.379514:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715718863190 time 715786446073 state 0x2 ref 0 00000100:00100000:3.0:1789760737.382564:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760737.382591:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678464 00000100:00100000:0.0:1789760737.382602:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678464 00000100:00100000:0.0:1789760737.382652:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715718863190 time 715786446073 state 0x0 ref 1 00000100:00100000:0.0:1789760737.382667:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715786706489 time 715786446073 state 0x1 ref 1 00000100:00100000:0.0:1789760737.382698:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715786706489 time 715847987355 state 0x0 ref 1 00000100:00100000:0.0:1789760737.382898:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515678464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760737.382946:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21757952, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:0.0:1789760737.382959:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.382976:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760737.382980:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.382986:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.382999:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.428207:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972629 is committed 00002000:00010000:2.0:1789760737.428438:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21757952 -> 21762048 00010000:00010000:2.0:1789760737.428453:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.428470:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.428478:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.428485:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.428490:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.428499:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.428518:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.428526:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.428614:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515678464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45901us (46055us total) trans 4294972629 rc 0/0 00000100:00100000:2.0:1789760737.428623:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5341 00000100:00100000:2.0:1789760737.428632:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715786706489 time 715847987355 state 0x2 ref 0 00000100:00100000:0.0:1789760737.436561:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760737.436704:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678592 00000100:00100000:1.0:1789760737.436792:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678592 00000100:00100000:0.0:1789760737.440567:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715786706489 time 715847987355 state 0x0 ref 1 00000100:00100000:0.0:1789760737.440583:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715848247771 time 715847987355 state 0x1 ref 1 00000100:00100000:0.0:1789760737.440619:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715848247771 time 715905908675 state 0x0 ref 1 00000100:00100000:0.0:1789760737.440631:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515678592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760737.440686:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21762048, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:0.0:1789760737.440695:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.440724:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760737.440729:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.440734:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.440748:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.500498:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972630 is committed 00002000:00010000:0.0:1789760737.500760:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21762048 -> 21766144 00010000:00010000:0.0:1789760737.500776:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.500789:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.500799:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.500806:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760737.500811:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.500821:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.500840:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.500847:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e3b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.501406:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515678592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60303us (64379us total) trans 4294972630 rc 0/0 00000100:00100000:0.0:1789760737.501420:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5342 00000100:00100000:0.0:1789760737.501431:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715848247771 time 715905908675 state 0x2 ref 0 00000100:00100000:3.0:1789760737.507900:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760737.507968:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678720 00000100:00100000:2.0:1789760737.507978:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678720 00000100:00100000:2.0:1789760737.508040:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715848247771 time 715905908675 state 0x0 ref 1 00000100:00100000:2.0:1789760737.508055:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715906169091 time 715905908675 state 0x1 ref 1 00000100:00100000:2.0:1789760737.508099:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715906169091 time 715973388136 state 0x0 ref 1 00000100:00100000:2.0:1789760737.508112:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515678720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760737.508167:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21766144, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:2.0:1789760737.508177:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.508193:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760737.508197:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.508203:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.508216:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.543673:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21766144 -> 21770240 00010000:00010000:2.0:1789760737.543694:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.543727:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.543743:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.543751:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.543756:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.543766:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.543790:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.543799:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.543895:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515678720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35780us (35999us total) trans 4294972631 rc 0/0 00000100:00100000:2.0:1789760737.543906:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5343 00000100:00100000:2.0:1789760737.543915:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715906169091 time 715973388136 state 0x2 ref 0 00000001:00080000:3.0:1789760737.547653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972631 is committed 00000100:00100000:0.0:1789760737.548073:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.548153:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678848 00000100:00100000:3.0:1789760737.548162:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678848 00000100:00100000:3.0:1789760737.548217:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715906169091 time 715973388136 state 0x0 ref 1 00000100:00100000:3.0:1789760737.548233:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715973648552 time 715973388136 state 0x1 ref 1 00000100:00100000:3.0:1789760737.548263:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715973648552 time 716013551918 state 0x0 ref 1 00000100:00100000:3.0:1789760737.548276:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515678848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.548323:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21770240, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:3.0:1789760737.548332:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.548349:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.548353:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.548360:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.548374:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.572239:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972632 is committed 00002000:00010000:3.0:1789760737.572300:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21770240 -> 21774336 00010000:00010000:3.0:1789760737.572314:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.572326:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.572336:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.572343:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760737.572347:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.572358:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.572375:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.572381:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.572790:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515678848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24511us (24717us total) trans 4294972632 rc 0/0 00000100:00100000:3.0:1789760737.572800:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5344 00000100:00100000:3.0:1789760737.572809:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715973648552 time 716013551918 state 0x2 ref 0 00000100:00100000:1.0:1789760737.576498:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.576583:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515678976 00000100:00100000:3.0:1789760737.576593:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515678976 00000100:00100000:3.0:1789760737.576651:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 715973648552 time 716013551918 state 0x0 ref 1 00000100:00100000:3.0:1789760737.576667:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716013812334 time 716013551918 state 0x1 ref 1 00000100:00100000:3.0:1789760737.576768:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716013812334 time 716042056776 state 0x0 ref 1 00000100:00100000:3.0:1789760737.576781:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515678976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.576835:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21774336, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:3.0:1789760737.576844:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.576860:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.576864:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.576871:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.576886:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.597831:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760737.612298:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21774336 -> 21778432 00010000:00010000:3.0:1789760737.612319:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.612332:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.612342:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.612350:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760737.612355:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.612364:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.612384:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.612393:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.612475:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515678976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35692us (35982us total) trans 4294972633 rc 0/0 00000100:00100000:3.0:1789760737.612485:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5345 00000100:00100000:3.0:1789760737.612495:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716013812334 time 716042056776 state 0x2 ref 0 00000001:00080000:3.0:1789760737.612532:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972633 is committed 00000100:00100000:3.0:1789760737.618829:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760737.619308:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679104 00000100:00100000:1.0:1789760737.619318:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679104 00000100:00100000:1.0:1789760737.619371:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716013812334 time 716042056776 state 0x0 ref 1 00000100:00100000:1.0:1789760737.619387:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716042317192 time 716042056776 state 0x1 ref 1 00000100:00100000:1.0:1789760737.619415:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716042317192 time 716084703112 state 0x0 ref 1 00000100:00100000:1.0:1789760737.619428:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515679104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760737.619479:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21778432, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:1.0:1789760737.619488:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.619504:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760737.619507:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.619513:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.619532:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.702459:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972634 is committed 00002000:00010000:2.0:1789760737.702575:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21778432 -> 21782528 00010000:00010000:2.0:1789760737.702591:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702609:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702618:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702626:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760737.702631:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702639:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702658:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702664:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e3cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.702822:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515679104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 83391us (84064us total) trans 4294972634 rc 0/0 00000100:00100000:2.0:1789760737.702833:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5346 00000100:00100000:2.0:1789760737.702842:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716042317192 time 716084703112 state 0x2 ref 0 00000100:00100000:0.0:1789760737.713099:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760737.713182:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679232 00000100:00100000:2.0:1789760737.713193:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679232 00000100:00100000:2.0:1789760737.713243:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716042317192 time 716084703112 state 0x0 ref 1 00000100:00100000:2.0:1789760737.713259:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716084963528 time 716084703112 state 0x1 ref 1 00000100:00100000:2.0:1789760737.713291:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716084963528 time 716178579593 state 0x0 ref 1 00000100:00100000:2.0:1789760737.713303:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515679232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760737.713348:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21782528, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:2.0:1789760737.713356:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.713371:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.713376:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.713382:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.713397:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.755058:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972635 is committed 00002000:00010000:3.0:1789760737.755155:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21782528 -> 21786624 00010000:00010000:3.0:1789760737.755170:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.755183:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.755191:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.755198:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760737.755203:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.755212:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.755232:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.755240:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e3d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.755333:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515679232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42026us (42239us total) trans 4294972635 rc 0/0 00000100:00100000:3.0:1789760737.755343:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5347 00000100:00100000:3.0:1789760737.755352:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716084963528 time 716178579593 state 0x2 ref 0 00000100:00100000:1.0:1789760737.764623:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760737.764754:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679360 00000100:00100000:0.0:1789760737.764763:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679360 00000100:00100000:0.0:1789760737.764813:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716084963528 time 716178579593 state 0x0 ref 1 00000100:00100000:0.0:1789760737.764828:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716178840009 time 716178579593 state 0x1 ref 1 00000100:00100000:0.0:1789760737.764863:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716178840009 time 716230152605 state 0x0 ref 1 00000100:00100000:0.0:1789760737.764876:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515679360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760737.764920:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21786624, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:0.0:1789760737.764940:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.764956:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.764961:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.764968:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.764983:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760737.807221:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21786624 -> 21790720 00010000:00010000:0.0:1789760737.807242:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.807256:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.807267:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.807275:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760737.807280:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.807289:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.807309:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.807317:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.807406:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515679360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42528us (42788us total) trans 4294972636 rc 0/0 00000100:00100000:0.0:1789760737.807417:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5348 00000100:00100000:0.0:1789760737.807428:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716178840009 time 716230152605 state 0x2 ref 0 00000001:00080000:3.0:1789760737.807561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972636 is committed 00000100:00100000:2.0:1789760737.810859:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.811002:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679488 00000100:00100000:3.0:1789760737.811021:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679488 00000100:00100000:3.0:1789760737.811074:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716178840009 time 716230152605 state 0x0 ref 1 00000100:00100000:3.0:1789760737.811087:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716230413021 time 716230152605 state 0x1 ref 1 00000100:00100000:3.0:1789760737.811156:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716230413021 time 716276444544 state 0x0 ref 1 00000100:00100000:3.0:1789760737.811170:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515679488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.811218:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21790720, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa807 00010000:00010000:3.0:1789760737.811227:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.811247:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.811251:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.811257:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.811270:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.861582:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21790720 -> 21794816 00002000:00010000:3.0:1789760737.861591:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760737 -> 1789760738 00002000:00010000:3.0:1789760737.861594:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760737 -> 1789760738 00010000:00010000:3.0:1789760737.861608:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.861630:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.861639:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.861646:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760737.861745:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.861758:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.861778:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.861786:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e3e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.861792:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972637 is committed 00000100:00100000:3.0:1789760737.861881:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515679488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50708us (51027us total) trans 4294972637 rc 0/0 00000100:00100000:3.0:1789760737.861891:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5349 00000100:00100000:3.0:1789760737.861901:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716230413021 time 716276444544 state 0x2 ref 0 00000100:00100000:2.0:1789760737.866051:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.866108:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679616 00000100:00100000:3.0:1789760737.866119:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679616 00000100:00100000:3.0:1789760737.866174:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716230413021 time 716276444544 state 0x0 ref 1 00000100:00100000:3.0:1789760737.866190:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716276704960 time 716276444544 state 0x1 ref 1 00000100:00100000:3.0:1789760737.866229:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716276704960 time 716331517824 state 0x0 ref 1 00000100:00100000:3.0:1789760737.866243:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515679616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.866293:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21794816, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760737.866303:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.866318:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.866322:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.866329:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.866343:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.937382:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972638 is committed 00002000:00010000:3.0:1789760737.937447:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21794816 -> 21798912 00010000:00010000:3.0:1789760737.937461:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.937474:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.937484:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.937492:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760737.937496:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.937505:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.937524:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.937537:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e3e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.937615:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515679616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71371us (71576us total) trans 4294972638 rc 0/0 00000100:00100000:3.0:1789760737.937624:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5350 00000100:00100000:3.0:1789760737.937634:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716276704960 time 716331517824 state 0x2 ref 0 00000100:00100000:1.0:1789760737.942679:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760737.942799:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679744 00000100:00100000:3.0:1789760737.942809:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679744 00000100:00100000:3.0:1789760737.943105:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716276704960 time 716331517824 state 0x0 ref 1 00000100:00100000:3.0:1789760737.943123:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716331778240 time 716331517824 state 0x1 ref 1 00000100:00100000:3.0:1789760737.943153:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716331778240 time 716408442524 state 0x0 ref 1 00000100:00100000:3.0:1789760737.943166:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515679744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760737.943218:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21798912, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760737.943226:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.943243:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.943247:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.943253:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.943271:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.997155:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972639 is committed 00002000:00010000:0.0:1789760737.997257:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21798912 -> 21803008 00010000:00010000:0.0:1789760737.997273:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.997288:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.997298:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.997305:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760737.997311:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.997320:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.997340:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.997349:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e3f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.997444:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515679744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54275us (54768us total) trans 4294972639 rc 0/0 00000100:00100000:0.0:1789760737.997454:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5351 00000100:00100000:0.0:1789760737.997465:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716331778240 time 716408442524 state 0x2 ref 0 00000100:00100000:3.0:1789760738.000685:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760738.000853:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515679872 00000100:00100000:2.0:1789760738.000862:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515679872 00000100:00100000:2.0:1789760738.000928:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716331778240 time 716408442524 state 0x0 ref 1 00000100:00100000:2.0:1789760738.000950:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716408702940 time 716408442524 state 0x1 ref 1 00000100:00100000:2.0:1789760738.001053:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716408702940 time 716466341155 state 0x0 ref 1 00000100:00100000:2.0:1789760738.001067:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515679872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760738.001110:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21803008, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:2.0:1789760738.001119:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.001135:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760738.001139:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.001146:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.001161:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760738.001728:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760738.037445:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21803008 -> 21807104 00010000:00010000:2.0:1789760738.037467:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.037480:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.037489:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.037497:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760738.037503:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.037512:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.037532:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.037540:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e3f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.037634:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515679872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36564us (36951us total) trans 4294972640 rc 0/0 00000100:00100000:2.0:1789760738.037646:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5352 00000100:00100000:2.0:1789760738.037771:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716408702940 time 716466341155 state 0x2 ref 0 00000001:00080000:0.0:1789760738.045965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972640 is committed 00000100:00100000:3.0:1789760738.046922:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.047240:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680000 00000100:00100000:0.0:1789760738.047250:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680000 00000100:00100000:0.0:1789760738.047305:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716408702940 time 716466341155 state 0x0 ref 1 00000100:00100000:0.0:1789760738.047320:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716466601571 time 716466341155 state 0x1 ref 1 00000100:00100000:0.0:1789760738.047347:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716466601571 time 716512636072 state 0x0 ref 1 00000100:00100000:0.0:1789760738.047360:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515680000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.047406:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21807104, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:0.0:1789760738.047416:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.047440:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760738.047444:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.047451:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.047464:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.089328:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972641 is committed 00002000:00010000:1.0:1789760738.089494:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21807104 -> 21811200 00010000:00010000:1.0:1789760738.089509:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.089522:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.089530:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.089536:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760738.089541:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.089549:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.089568:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.089582:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e3fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.090743:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515680000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43342us (43784us total) trans 4294972641 rc 0/0 00000100:00100000:1.0:1789760738.090758:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5353 00000100:00100000:1.0:1789760738.090768:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716466601571 time 716512636072 state 0x2 ref 0 00000100:00100000:2.0:1789760738.096790:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.097353:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680128 00000100:00100000:0.0:1789760738.097363:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680128 00000100:00100000:0.0:1789760738.097417:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716466601571 time 716512636072 state 0x0 ref 1 00000100:00100000:0.0:1789760738.097434:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716512896488 time 716512636072 state 0x1 ref 1 00000100:00100000:0.0:1789760738.097469:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716512896488 time 716562758103 state 0x0 ref 1 00000100:00100000:0.0:1789760738.097482:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515680128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.097535:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21811200, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:0.0:1789760738.097543:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.097558:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760738.097562:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.097569:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.097582:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.193134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972642 is committed 00002000:00010000:3.0:1789760738.193237:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21811200 -> 21815296 00010000:00010000:3.0:1789760738.193252:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.193265:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.193273:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.193281:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.193286:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.193296:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.193315:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.193329:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e405 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.193454:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515680128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 95970us (96671us total) trans 4294972642 rc 0/0 00000100:00100000:3.0:1789760738.193466:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5354 00000100:00100000:3.0:1789760738.193475:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716512896488 time 716562758103 state 0x2 ref 0 00000100:00100000:3.0:1789760738.196389:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.196481:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680256 00000100:00100000:0.0:1789760738.196492:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680256 00000100:00100000:0.0:1789760738.196545:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716512896488 time 716562758103 state 0x0 ref 1 00000100:00100000:0.0:1789760738.196560:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716563018519 time 716562758103 state 0x1 ref 1 00000100:00100000:0.0:1789760738.196593:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716563018519 time 716661882064 state 0x0 ref 1 00000100:00100000:0.0:1789760738.196605:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515680256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.196653:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21815296, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:0.0:1789760738.196789:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.196806:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760738.196811:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.196818:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.196832:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760738.252146:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21815296 -> 21819392 00010000:00010000:0.0:1789760738.252164:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.252179:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.252189:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.252197:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760738.252202:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.252212:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.252238:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.252246:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e40c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.252334:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515680256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55726us (55951us total) trans 4294972643 rc 0/0 00000100:00100000:0.0:1789760738.252345:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5355 00000100:00100000:0.0:1789760738.252355:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716563018519 time 716661882064 state 0x2 ref 0 00000001:00080000:0.0:1789760738.252814:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972643 is committed 00000100:00100000:3.0:1789760738.255395:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760738.255533:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680384 00000100:00100000:2.0:1789760738.255543:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680384 00000100:00100000:2.0:1789760738.255594:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716563018519 time 716661882064 state 0x0 ref 1 00000100:00100000:2.0:1789760738.255610:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716662142480 time 716661882064 state 0x1 ref 1 00000100:00100000:2.0:1789760738.255640:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716662142480 time 716720929399 state 0x0 ref 1 00000100:00100000:2.0:1789760738.255661:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760738.255731:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21819392, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:2.0:1789760738.255740:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.255756:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760738.255760:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.255766:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.255782:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.290162:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972644 is committed 00002000:00010000:3.0:1789760738.290260:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21819392 -> 21823488 00010000:00010000:3.0:1789760738.290276:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.290289:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.290298:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.290305:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.290310:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.290320:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.290641:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.290781:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e413 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.290908:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35252us (35517us total) trans 4294972644 rc 0/0 00000100:00100000:3.0:1789760738.290919:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5356 00000100:00100000:3.0:1789760738.290988:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716662142480 time 716720929399 state 0x2 ref 0 00000100:00100000:2.0:1789760738.294467:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.294547:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680512 00000100:00100000:3.0:1789760738.294557:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680512 00000100:00100000:3.0:1789760738.294606:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716662142480 time 716720929399 state 0x0 ref 1 00000100:00100000:3.0:1789760738.294621:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716721189815 time 716720929399 state 0x1 ref 1 00000100:00100000:3.0:1789760738.294688:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716721189815 time 716759976798 state 0x0 ref 1 00000100:00100000:3.0:1789760738.294701:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.294774:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21823488, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760738.294784:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.294801:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760738.294806:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.294812:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.294826:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.320473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972645 is committed 00002000:00010000:3.0:1789760738.320601:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21823488 -> 21827584 00010000:00010000:3.0:1789760738.320616:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.320628:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.320637:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.320643:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.320743:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.320755:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.320773:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.320786:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e41a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.320868:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26164us (26405us total) trans 4294972645 rc 0/0 00000100:00100000:3.0:1789760738.320878:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5357 00000100:00100000:3.0:1789760738.320889:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716721189815 time 716759976798 state 0x2 ref 0 00000100:00100000:2.0:1789760738.325332:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.325809:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680640 00000100:00100000:1.0:1789760738.325820:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680640 00000100:00100000:1.0:1789760738.325876:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716721189815 time 716759976798 state 0x0 ref 1 00000100:00100000:1.0:1789760738.325891:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716760237214 time 716759976798 state 0x1 ref 1 00000100:00100000:1.0:1789760738.325977:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716760237214 time 716791264976 state 0x0 ref 1 00000100:00100000:1.0:1789760738.325992:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760738.326047:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21827584, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:1.0:1789760738.326056:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.326072:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760738.326077:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.326083:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.326097:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.349038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972646 is committed 00002000:00010000:1.0:1789760738.349111:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21827584 -> 21831680 00010000:00010000:1.0:1789760738.349128:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.349142:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.349152:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.349161:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760738.349166:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.349176:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.349196:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.349204:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e421 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.349316:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23323us (23989us total) trans 4294972646 rc 0/0 00000100:00100000:1.0:1789760738.349327:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5358 00000100:00100000:1.0:1789760738.349337:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716760237214 time 716791264976 state 0x2 ref 0 00000100:00100000:1.0:1789760738.353085:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.353176:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680768 00000100:00100000:0.0:1789760738.353186:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680768 00000100:00100000:0.0:1789760738.353241:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716760237214 time 716791264976 state 0x0 ref 1 00000100:00100000:0.0:1789760738.353257:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716791525392 time 716791264976 state 0x1 ref 1 00000100:00100000:0.0:1789760738.353287:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716791525392 time 716818576077 state 0x0 ref 1 00000100:00100000:0.0:1789760738.353300:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.353345:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21831680, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:0.0:1789760738.353353:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.353375:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760738.353380:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.353386:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.353400:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.394822:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972647 is committed 00002000:00010000:2.0:1789760738.394917:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21831680 -> 21835776 00010000:00010000:2.0:1789760738.394942:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.394956:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.394965:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.394972:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760738.394977:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.394987:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.395102:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.395112:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e428 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.395202:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41899us (42120us total) trans 4294972647 rc 0/0 00000100:00100000:2.0:1789760738.395213:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5359 00000100:00100000:2.0:1789760738.395223:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716791525392 time 716818576077 state 0x2 ref 0 00000100:00100000:2.0:1789760738.398264:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.398348:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515680896 00000100:00100000:3.0:1789760738.398357:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515680896 00000100:00100000:3.0:1789760738.398408:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716791525392 time 716818576077 state 0x0 ref 1 00000100:00100000:3.0:1789760738.398424:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716818836493 time 716818576077 state 0x1 ref 1 00000100:00100000:3.0:1789760738.398459:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716818836493 time 716863748031 state 0x0 ref 1 00000100:00100000:3.0:1789760738.398470:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.398522:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21835776, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760738.398532:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.398548:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760738.398552:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.398559:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.398575:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.424619:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972648 is committed 00002000:00010000:2.0:1789760738.424881:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21835776 -> 21839872 00010000:00010000:2.0:1789760738.424895:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.424908:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.424917:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.424940:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760738.424944:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.424952:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.424971:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.424978:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e42f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.425071:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515680896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26597us (26812us total) trans 4294972648 rc 0/0 00000100:00100000:2.0:1789760738.425080:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5360 00000100:00100000:2.0:1789760738.425094:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716818836493 time 716863748031 state 0x2 ref 0 00000100:00100000:3.0:1789760738.428210:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.428269:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681024 00000100:00100000:0.0:1789760738.428278:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681024 00000100:00100000:0.0:1789760738.428323:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716818836493 time 716863748031 state 0x0 ref 1 00000100:00100000:0.0:1789760738.428337:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716864008447 time 716863748031 state 0x1 ref 1 00000100:00100000:0.0:1789760738.428366:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716864008447 time 716893655809 state 0x0 ref 1 00000100:00100000:0.0:1789760738.428377:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515681024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.428420:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21839872, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:0.0:1789760738.428429:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.428445:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760738.428449:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.428456:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.428474:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760738.452489:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21839872 -> 21843968 00010000:00010000:3.0:1789760738.452509:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.452522:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.452531:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.452538:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.452542:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.452551:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.452569:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.452576:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e436 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.452770:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515681024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24389us (24563us total) trans 4294972649 rc 0/0 00000100:00100000:3.0:1789760738.452781:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5361 00000100:00100000:3.0:1789760738.452791:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716864008447 time 716893655809 state 0x2 ref 0 00000001:00080000:3.0:1789760738.452828:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972649 is committed 00000100:00100000:2.0:1789760738.455867:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.456012:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681152 00000100:00100000:3.0:1789760738.456022:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681152 00000100:00100000:3.0:1789760738.456077:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716864008447 time 716893655809 state 0x0 ref 1 00000100:00100000:3.0:1789760738.456094:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716893916225 time 716893655809 state 0x1 ref 1 00000100:00100000:3.0:1789760738.456157:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716893916225 time 716921446246 state 0x0 ref 1 00000100:00100000:3.0:1789760738.456169:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515681152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.456215:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21843968, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760738.456225:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.456241:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760738.456245:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.456251:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.456263:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760738.484545:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21843968 -> 21848064 00010000:00010000:3.0:1789760738.484562:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.484576:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.484584:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.484592:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.484597:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.484606:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.484625:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.484632:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e43d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.484755:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515681152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28583us (28892us total) trans 4294972650 rc 0/0 00000100:00100000:3.0:1789760738.484769:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5362 00000100:00100000:3.0:1789760738.484779:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716893916225 time 716921446246 state 0x2 ref 0 00000100:00100000:3.0:1789760738.488151:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.488343:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681280 00000100:00100000:0.0:1789760738.488353:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681280 00000100:00100000:0.0:1789760738.488404:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716893916225 time 716921446246 state 0x0 ref 1 00000100:00100000:0.0:1789760738.488419:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716921706662 time 716921446246 state 0x1 ref 1 00000100:00100000:0.0:1789760738.488454:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716921706662 time 716953742689 state 0x0 ref 1 00000100:00100000:0.0:1789760738.488466:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515681280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.488511:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21848064, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:0.0:1789760738.488521:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.488536:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760738.488539:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.488545:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.488558:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.490119:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972650 is committed 00002000:00010000:0.0:1789760738.510467:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21848064 -> 21852160 00010000:00010000:0.0:1789760738.510487:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.510501:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.510511:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.510519:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760738.510524:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.510533:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.510553:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.510562:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e444 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.510820:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515681280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22178us (22500us total) trans 4294972651 rc 0/0 00000100:00100000:0.0:1789760738.510832:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5363 00000100:00100000:0.0:1789760738.510842:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716921706662 time 716953742689 state 0x2 ref 0 00000001:00080000:0.0:1789760738.510882:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972651 is committed 00000100:00100000:3.0:1789760738.515376:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.515523:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681408 00000100:00100000:1.0:1789760738.515533:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681408 00000100:00100000:1.0:1789760738.515583:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716921706662 time 716953742689 state 0x0 ref 1 00000100:00100000:1.0:1789760738.515598:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716954003105 time 716953742689 state 0x1 ref 1 00000100:00100000:1.0:1789760738.515635:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716954003105 time 716980924195 state 0x0 ref 1 00000100:00100000:1.0:1789760738.515647:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515681408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760738.515700:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21852160, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:1.0:1789760738.515741:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.515759:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760738.515764:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.515770:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.515786:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.542556:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972652 is committed 00002000:00010000:3.0:1789760738.542596:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21852160 -> 21856256 00010000:00010000:3.0:1789760738.542622:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.542635:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.542652:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.542660:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.542665:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.542676:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.542698:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.542727:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e44b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.542820:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515681408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27169us (27448us total) trans 4294972652 rc 0/0 00000100:00100000:3.0:1789760738.542831:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5364 00000100:00100000:3.0:1789760738.542841:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716954003105 time 716980924195 state 0x2 ref 0 00000100:00100000:0.0:1789760738.549793:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.549936:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681536 00000100:00100000:1.0:1789760738.549950:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681536 00000100:00100000:1.0:1789760738.550002:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716954003105 time 716980924195 state 0x0 ref 1 00000100:00100000:1.0:1789760738.550026:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716981184611 time 716980924195 state 0x1 ref 1 00000100:00100000:1.0:1789760738.550057:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716981184611 time 717015346019 state 0x0 ref 1 00000100:00100000:1.0:1789760738.550069:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515681536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760738.550114:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21856256, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:1.0:1789760738.550124:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.550139:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.550143:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.550149:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.550162:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.582877:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972653 is committed 00002000:00010000:3.0:1789760738.583040:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21856256 -> 21860352 00010000:00010000:3.0:1789760738.583055:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.583069:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.583079:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.583087:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.583092:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.583102:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.583122:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.583136:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e452 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.583260:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515681536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33188us (33473us total) trans 4294972653 rc 0/0 00000100:00100000:3.0:1789760738.583270:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5365 00000100:00100000:3.0:1789760738.583279:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716981184611 time 717015346019 state 0x2 ref 0 00000100:00100000:0.0:1789760738.586336:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.586445:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681664 00000100:00100000:1.0:1789760738.586454:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681664 00000100:00100000:1.0:1789760738.586504:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 716981184611 time 717015346019 state 0x0 ref 1 00000100:00100000:1.0:1789760738.586523:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717015606435 time 717015346019 state 0x1 ref 1 00000100:00100000:1.0:1789760738.586583:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717015606435 time 717051871589 state 0x0 ref 1 00000100:00100000:1.0:1789760738.586596:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515681664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760738.586643:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21860352, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:1.0:1789760738.586749:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.586767:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.586772:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.586778:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.586793:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.611327:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972654 is committed 00002000:00010000:2.0:1789760738.611419:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21860352 -> 21864448 00010000:00010000:2.0:1789760738.611434:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611448:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611459:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611467:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760738.611473:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611483:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611503:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611512:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e459 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.611633:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515681664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25033us (25300us total) trans 4294972654 rc 0/0 00000100:00100000:2.0:1789760738.611643:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5366 00000100:00100000:2.0:1789760738.611663:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717015606435 time 717051871589 state 0x2 ref 0 00000100:00100000:0.0:1789760738.615958:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.616093:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681792 00000100:00100000:3.0:1789760738.616102:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681792 00000100:00100000:3.0:1789760738.616151:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717015606435 time 717051871589 state 0x0 ref 1 00000100:00100000:3.0:1789760738.616166:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717052132005 time 717051871589 state 0x1 ref 1 00000100:00100000:3.0:1789760738.616198:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717052132005 time 717081486933 state 0x0 ref 1 00000100:00100000:3.0:1789760738.616210:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515681792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.616255:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21864448, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760738.616264:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.616281:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.616285:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.616291:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.616311:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.624799:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760738.642546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972655 is committed 00002000:00010000:3.0:1789760738.642634:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21864448 -> 21868544 00010000:00010000:3.0:1789760738.642649:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.642662:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.642671:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.642678:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.642683:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.642692:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.642728:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.642737:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e460 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.642828:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515681792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26615us (26875us total) trans 4294972655 rc 0/0 00000100:00100000:3.0:1789760738.642838:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5367 00000100:00100000:3.0:1789760738.642848:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717052132005 time 717081486933 state 0x2 ref 0 00000100:00100000:0.0:1789760738.645885:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.646018:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515681920 00000100:00100000:3.0:1789760738.646027:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515681920 00000100:00100000:3.0:1789760738.646076:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717052132005 time 717081486933 state 0x0 ref 1 00000100:00100000:3.0:1789760738.646092:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717081747349 time 717081486933 state 0x1 ref 1 00000100:00100000:3.0:1789760738.646123:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717081747349 time 717111412739 state 0x0 ref 1 00000100:00100000:3.0:1789760738.646135:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515681920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.646180:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21868544, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760738.646188:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.646204:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.646209:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.646215:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.646229:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.673692:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972656 is committed 00002000:00010000:3.0:1789760738.673917:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21868544 -> 21872640 00010000:00010000:3.0:1789760738.673933:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.673950:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.673960:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.673967:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.673972:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.673982:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674001:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674019:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e467 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.674275:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515681920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28136us (28394us total) trans 4294972656 rc 0/0 00000100:00100000:3.0:1789760738.674289:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5368 00000100:00100000:3.0:1789760738.674299:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717081747349 time 717111412739 state 0x2 ref 0 00000100:00100000:0.0:1789760738.678615:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760738.678688:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682048 00000100:00100000:2.0:1789760738.678698:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682048 00000100:00100000:2.0:1789760738.678791:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717081747349 time 717111412739 state 0x0 ref 1 00000100:00100000:2.0:1789760738.678808:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717111673155 time 717111412739 state 0x1 ref 1 00000100:00100000:2.0:1789760738.678839:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717111673155 time 717144128888 state 0x0 ref 1 00000100:00100000:2.0:1789760738.678852:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760738.678899:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21872640, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:2.0:1789760738.678908:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.678935:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.678939:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.678946:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.678960:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.719324:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972657 is committed 00002000:00010000:0.0:1789760738.719411:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21872640 -> 21876736 00010000:00010000:0.0:1789760738.719429:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.719441:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.719449:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.719456:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760738.719460:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.719469:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.719494:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.719501:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e46e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.719794:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40938us (41183us total) trans 4294972657 rc 0/0 00000100:00100000:0.0:1789760738.719805:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5369 00000100:00100000:0.0:1789760738.719815:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717111673155 time 717144128888 state 0x2 ref 0 00000100:00100000:0.0:1789760738.724443:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.724557:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682176 00000100:00100000:1.0:1789760738.724567:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682176 00000100:00100000:2.0:1789760738.724966:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717111673155 time 717144128888 state 0x0 ref 1 00000100:00100000:2.0:1789760738.724982:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717144389304 time 717144128888 state 0x1 ref 1 00000100:00100000:2.0:1789760738.725022:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717144389304 time 717190310745 state 0x0 ref 1 00000100:00100000:2.0:1789760738.725035:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515682176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760738.725081:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21876736, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:2.0:1789760738.725189:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.725272:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.725276:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.725282:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.725296:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760738.760364:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21876736 -> 21880832 00010000:00010000:3.0:1789760738.760382:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.760393:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.760399:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.760403:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.760407:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.760413:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.760428:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.760432:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e475 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.760511:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515682176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35474us (36075us total) trans 4294972658 rc 0/0 00000100:00100000:3.0:1789760738.760522:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5370 00000100:00100000:3.0:1789760738.760531:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717144389304 time 717190310745 state 0x2 ref 0 00000001:00080000:3.0:1789760738.760571:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972658 is committed 00000100:00100000:2.0:1789760738.766332:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.766798:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682304 00000100:00100000:1.0:1789760738.766809:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682304 00000100:00100000:1.0:1789760738.766861:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717144389304 time 717190310745 state 0x0 ref 1 00000100:00100000:1.0:1789760738.766875:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717190571161 time 717190310745 state 0x1 ref 1 00000100:00100000:1.0:1789760738.766942:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717190571161 time 717232231432 state 0x0 ref 1 00000100:00100000:1.0:1789760738.766952:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515682304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760738.766991:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21880832, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:1.0:1789760738.766998:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.767022:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.767030:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.767036:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.767051:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.793235:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972659 is committed 00002000:00010000:3.0:1789760738.793330:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21880832 -> 21884928 00010000:00010000:3.0:1789760738.793344:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.793356:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.793366:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.793378:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.793383:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.793392:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.793410:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.793423:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e47c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.793533:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515682304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26578us (27206us total) trans 4294972659 rc 0/0 00000100:00100000:3.0:1789760738.793541:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5371 00000100:00100000:3.0:1789760738.793548:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717190571161 time 717232231432 state 0x2 ref 0 00000100:00100000:3.0:1789760738.796629:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760738.797021:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682432 00000100:00100000:1.0:1789760738.797031:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682432 00000100:00100000:1.0:1789760738.797084:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717190571161 time 717232231432 state 0x0 ref 1 00000100:00100000:1.0:1789760738.797099:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717232491848 time 717232231432 state 0x1 ref 1 00000100:00100000:1.0:1789760738.797134:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717232491848 time 717262423215 state 0x0 ref 1 00000100:00100000:1.0:1789760738.797158:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760738.797205:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21884928, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:1.0:1789760738.797214:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.797236:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760738.797240:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.797246:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.797261:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.814769:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760738.815321:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544411776:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760738.815397:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760738.815464:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411776 00000100:00100000:2.0:1789760738.815471:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411776 00000100:00100000:2.0:1789760738.815521:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:2.0:1789760738.815530:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544411776:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760738.815561:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760738.815602:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544411776:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760738.815609:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00080000:2.0:1789760738.815628:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760738.815660:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544411776:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760738.815805:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c760700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544411648:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760738.815840:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760738.816685:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411648 00000100:00100000:2.0:1789760738.816694:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411648 00000100:00100000:2.0:1789760738.816749:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760738.816756:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544411648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760738.816774:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3836739584 avail 3805282304 00000100:00100000:2.0:1789760738.816840:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544411648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (999us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760738.816847:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00080000:2.0:1789760738.816867:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760738.816919:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c760700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544411648:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760738.844643:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21884928 -> 21889024 00010000:00010000:2.0:1789760738.844665:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.844690:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.844699:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.844742:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760738.844748:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.844757:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.844776:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.844784:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e483 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.844864:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515682432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47715us (48239us total) trans 4294972660 rc 0/0 00000100:00100000:2.0:1789760738.844875:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5372 00000100:00100000:2.0:1789760738.844884:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717232491848 time 717262423215 state 0x2 ref 0 00000001:00080000:3.0:1789760738.844979:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972660 is committed 00000100:00100000:0.0:1789760738.851078:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.851234:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682560 00000100:00100000:3.0:1789760738.851244:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682560 00000100:00100000:3.0:1789760738.851295:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717232491848 time 717262423215 state 0x0 ref 1 00000100:00100000:3.0:1789760738.851310:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717262683631 time 717262423215 state 0x1 ref 1 00000100:00100000:3.0:1789760738.851381:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717262683631 time 717316668875 state 0x0 ref 1 00000100:00100000:3.0:1789760738.851394:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515682560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.851442:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21889024, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa807 00010000:00010000:3.0:1789760738.851452:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.851467:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.851473:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.851480:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.851495:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.883606:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972661 is committed 00002000:00010000:3.0:1789760738.883802:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21889024 -> 21893120 00002000:00010000:3.0:1789760738.883807:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760738 -> 1789760739 00002000:00010000:3.0:1789760738.883810:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760738 -> 1789760739 00010000:00010000:3.0:1789760738.883824:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.883838:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.883848:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.883857:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.883862:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.883872:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.883892:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.883906:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e48a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.884054:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515682560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32656us (32979us total) trans 4294972661 rc 0/0 00000100:00100000:3.0:1789760738.884065:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5373 00000100:00100000:3.0:1789760738.884074:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717262683631 time 717316668875 state 0x2 ref 0 00000100:00100000:0.0:1789760738.888486:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760738.888584:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682688 00000100:00100000:3.0:1789760738.888593:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682688 00000100:00100000:3.0:1789760738.888889:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717262683631 time 717316668875 state 0x0 ref 1 00000100:00100000:3.0:1789760738.888908:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717316929291 time 717316668875 state 0x1 ref 1 00000100:00100000:3.0:1789760738.889890:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717316929291 time 717355171964 state 0x0 ref 1 00000100:00100000:3.0:1789760738.889912:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760738.889978:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21893120, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760738.889987:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.890013:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.890018:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.890024:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.890044:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.934528:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972662 is committed 00002000:00010000:3.0:1789760738.934674:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21893120 -> 21897216 00010000:00010000:3.0:1789760738.934690:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.934703:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.934867:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.934875:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.934881:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.934891:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.934914:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.934930:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e491 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.935330:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45419us (46848us total) trans 4294972662 rc 0/0 00000100:00100000:2.0:1789760738.935341:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5374 00000100:00100000:2.0:1789760738.935350:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717316929291 time 717355171964 state 0x2 ref 0 00000100:00100000:1.0:1789760738.938635:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760738.938991:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682816 00000100:00100000:0.0:1789760738.939011:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682816 00000100:00100000:0.0:1789760738.939066:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717316929291 time 717355171964 state 0x0 ref 1 00000100:00100000:0.0:1789760738.939082:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717355432380 time 717355171964 state 0x1 ref 1 00000100:00100000:0.0:1789760738.939163:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717355432380 time 717404451776 state 0x0 ref 1 00000100:00100000:0.0:1789760738.939176:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760738.939236:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21897216, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760738.939245:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.939262:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760738.939268:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.939275:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.939289:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.969916:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972663 is committed 00002000:00010000:3.0:1789760738.970095:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21897216 -> 21901312 00010000:00010000:3.0:1789760738.970111:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970122:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970131:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970137:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760738.970142:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970151:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970171:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970178:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e498 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.970280:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515682816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31100us (31648us total) trans 4294972663 rc 0/0 00000100:00100000:3.0:1789760738.970295:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5375 00000100:00100000:3.0:1789760738.970305:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717355432380 time 717404451776 state 0x2 ref 0 00000100:00100000:1.0:1789760738.975198:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760738.975306:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515682944 00000100:00100000:2.0:1789760738.975317:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515682944 00000100:00100000:2.0:1789760738.975372:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717355432380 time 717404451776 state 0x0 ref 1 00000100:00100000:2.0:1789760738.975387:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717404712192 time 717404451776 state 0x1 ref 1 00000100:00100000:2.0:1789760738.975447:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717404712192 time 717440735237 state 0x0 ref 1 00000100:00100000:2.0:1789760738.975460:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515682944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760738.975513:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21901312, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:2.0:1789760738.975523:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.975541:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.975545:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.975553:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.975567:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.015293:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972664 is committed 00002000:00010000:0.0:1789760739.015356:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21901312 -> 21905408 00010000:00010000:0.0:1789760739.015372:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.015385:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.015393:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.015405:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.015410:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.015417:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.015436:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.015443:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e49f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.015544:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515682944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40082us (40352us total) trans 4294972664 rc 0/0 00000100:00100000:0.0:1789760739.015555:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5376 00000100:00100000:0.0:1789760739.015566:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717404712192 time 717440735237 state 0x2 ref 0 00000100:00100000:0.0:1789760739.019074:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.019234:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683072 00000100:00100000:3.0:1789760739.019245:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683072 00000100:00100000:3.0:1789760739.019300:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717404712192 time 717440735237 state 0x0 ref 1 00000100:00100000:3.0:1789760739.019315:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717440995653 time 717440735237 state 0x1 ref 1 00000100:00100000:3.0:1789760739.019342:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717440995653 time 717484631432 state 0x0 ref 1 00000100:00100000:3.0:1789760739.019355:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515683072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.019402:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21905408, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.019410:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.019426:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760739.019430:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.019436:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.019450:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.064351:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21905408 -> 21909504 00010000:00010000:0.0:1789760739.064371:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.064385:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.064399:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.064407:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.064412:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.064421:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.064441:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.064450:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e4a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.064544:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515683072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45186us (45473us total) trans 4294972665 rc 0/0 00000100:00100000:0.0:1789760739.064555:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5377 00000100:00100000:0.0:1789760739.064564:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717440995653 time 717484631432 state 0x2 ref 0 00000001:00080000:1.0:1789760739.066256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972665 is committed 00000100:00100000:2.0:1789760739.071171:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760739.071282:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683200 00000100:00100000:1.0:1789760739.071292:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683200 00000100:00100000:1.0:1789760739.071342:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717440995653 time 717484631432 state 0x0 ref 1 00000100:00100000:1.0:1789760739.071357:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717484891848 time 717484631432 state 0x1 ref 1 00000100:00100000:1.0:1789760739.071387:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717484891848 time 717536676514 state 0x0 ref 1 00000100:00100000:1.0:1789760739.071399:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760739.071446:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21909504, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:1.0:1789760739.071455:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.071471:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760739.071476:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.071482:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.071496:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.099433:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972666 is committed 00002000:00010000:0.0:1789760739.099750:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21909504 -> 21913600 00010000:00010000:0.0:1789760739.099766:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.099784:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.099794:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.099801:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.099806:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.099815:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.099834:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.099842:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e4ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.100101:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28699us (28935us total) trans 4294972666 rc 0/0 00000100:00100000:0.0:1789760739.100112:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5378 00000100:00100000:0.0:1789760739.100123:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717484891848 time 717536676514 state 0x2 ref 0 00000100:00100000:3.0:1789760739.102799:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.103050:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683328 00000100:00100000:0.0:1789760739.103061:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683328 00000100:00100000:0.0:1789760739.103118:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717484891848 time 717536676514 state 0x0 ref 1 00000100:00100000:0.0:1789760739.103132:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717536936930 time 717536676514 state 0x1 ref 1 00000100:00100000:0.0:1789760739.103164:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717536936930 time 717568453377 state 0x0 ref 1 00000100:00100000:0.0:1789760739.103176:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515683328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.103229:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21913600, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.103237:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.103252:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760739.103255:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.103262:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.103274:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.141604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972667 is committed 00002000:00010000:0.0:1789760739.141776:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21913600 -> 21917696 00010000:00010000:0.0:1789760739.141791:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.141804:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.141818:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.141825:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.141830:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.141837:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.141855:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.141862:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.141954:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515683328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38776us (39159us total) trans 4294972667 rc 0/0 00000100:00100000:0.0:1789760739.141964:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5379 00000100:00100000:0.0:1789760739.141973:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717536936930 time 717568453377 state 0x2 ref 0 00000100:00100000:2.0:1789760739.146171:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.146335:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683456 00000100:00100000:0.0:1789760739.146346:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683456 00000100:00100000:0.0:1789760739.146393:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717536936930 time 717568453377 state 0x0 ref 1 00000100:00100000:0.0:1789760739.146407:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717568713793 time 717568453377 state 0x1 ref 1 00000100:00100000:0.0:1789760739.146438:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717568713793 time 717611727015 state 0x0 ref 1 00000100:00100000:0.0:1789760739.146450:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515683456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.146496:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21917696, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.146505:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.146520:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760739.146524:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.146529:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.146543:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.175654:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21917696 -> 21921792 00010000:00010000:0.0:1789760739.175675:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.175687:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.175696:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.175703:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.175805:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.175815:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.175836:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.175843:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.175940:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515683456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29486us (29773us total) trans 4294972668 rc 0/0 00000100:00100000:0.0:1789760739.175951:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5380 00000100:00100000:0.0:1789760739.175961:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717568713793 time 717611727015 state 0x2 ref 0 00000001:00080000:2.0:1789760739.177184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972668 is committed 00000100:00100000:0.0:1789760739.180659:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.180865:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683584 00000100:00100000:3.0:1789760739.180874:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683584 00000100:00100000:3.0:1789760739.180944:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717568713793 time 717611727015 state 0x0 ref 1 00000100:00100000:3.0:1789760739.180960:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717611987431 time 717611727015 state 0x1 ref 1 00000100:00100000:3.0:1789760739.180986:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717611987431 time 717646275352 state 0x0 ref 1 00000100:00100000:3.0:1789760739.181008:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.181055:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21921792, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.181065:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.181081:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760739.181086:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.181092:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.181106:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.236349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972669 is committed 00002000:00010000:3.0:1789760739.236689:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21921792 -> 21925888 00010000:00010000:3.0:1789760739.236738:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.236751:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.236767:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.236776:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760739.236781:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.236791:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.236810:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.236820:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.236905:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55902us (56250us total) trans 4294972669 rc 0/0 00000100:00100000:3.0:1789760739.236915:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5381 00000100:00100000:3.0:1789760739.236939:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717611987431 time 717646275352 state 0x2 ref 0 00000100:00100000:1.0:1789760739.246428:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760739.246524:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683712 00000100:00100000:2.0:1789760739.246533:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683712 00000100:00100000:2.0:1789760739.246580:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717611987431 time 717646275352 state 0x0 ref 1 00000100:00100000:2.0:1789760739.246594:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717646535768 time 717646275352 state 0x1 ref 1 00000100:00100000:2.0:1789760739.246624:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717646535768 time 717711913312 state 0x0 ref 1 00000100:00100000:2.0:1789760739.246635:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.247199:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21925888, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.247210:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.247231:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760739.247235:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.247241:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.247255:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.287887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972670 is committed 00002000:00010000:0.0:1789760739.288048:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21925888 -> 21929984 00010000:00010000:0.0:1789760739.288064:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.288077:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.288086:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.288094:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.288099:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.288107:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.288125:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.288132:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e4c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.288224:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41585us (41798us total) trans 4294972670 rc 0/0 00000100:00100000:0.0:1789760739.288233:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5382 00000100:00100000:0.0:1789760739.288243:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717646535768 time 717711913312 state 0x2 ref 0 00000100:00100000:2.0:1789760739.296410:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.296655:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683840 00000100:00100000:3.0:1789760739.296665:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683840 00000100:00100000:3.0:1789760739.296762:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717646535768 time 717711913312 state 0x0 ref 1 00000100:00100000:3.0:1789760739.296777:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717712173728 time 717711913312 state 0x1 ref 1 00000100:00100000:3.0:1789760739.296798:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717712173728 time 717762087353 state 0x0 ref 1 00000100:00100000:3.0:1789760739.296810:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.296858:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21929984, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.296867:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.296883:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760739.296888:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.296894:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.296913:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.347379:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972671 is committed 00002000:00010000:3.0:1789760739.347556:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21929984 -> 21934080 00010000:00010000:3.0:1789760739.347579:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347592:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347600:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347608:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760739.347612:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347620:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347734:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.347743:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.347886:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51073us (51481us total) trans 4294972671 rc 0/0 00000100:00100000:3.0:1789760739.347896:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5383 00000100:00100000:3.0:1789760739.347906:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717712173728 time 717762087353 state 0x2 ref 0 00000100:00100000:1.0:1789760739.353048:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.353203:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515683968 00000100:00100000:0.0:1789760739.353214:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515683968 00000100:00100000:0.0:1789760739.353268:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717712173728 time 717762087353 state 0x0 ref 1 00000100:00100000:0.0:1789760739.353282:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717762347769 time 717762087353 state 0x1 ref 1 00000100:00100000:0.0:1789760739.353405:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717762347769 time 717818693359 state 0x0 ref 1 00000100:00100000:0.0:1789760739.353420:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.353563:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21934080, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.353833:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.353852:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760739.353857:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.353863:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.353878:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.387215:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972672 is committed 00002000:00010000:0.0:1789760739.387322:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21934080 -> 21938176 00010000:00010000:0.0:1789760739.387339:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.387352:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.387360:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.387369:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.387374:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.387383:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.387403:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.387411:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.387791:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515683968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34368us (34748us total) trans 4294972672 rc 0/0 00000100:00100000:0.0:1789760739.387802:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5384 00000100:00100000:0.0:1789760739.387812:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717762347769 time 717818693359 state 0x2 ref 0 00000100:00100000:1.0:1789760739.393283:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.393376:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684096 00000100:00100000:0.0:1789760739.393386:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684096 00000100:00100000:0.0:1789760739.393509:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717762347769 time 717818693359 state 0x0 ref 1 00000100:00100000:0.0:1789760739.393526:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717818953775 time 717818693359 state 0x1 ref 1 00000100:00100000:0.0:1789760739.393595:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717818953775 time 717858884105 state 0x0 ref 1 00000100:00100000:0.0:1789760739.393609:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515684096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.393664:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21938176, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.393674:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.393690:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760739.393694:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.393700:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.393759:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.424849:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972673 is committed 00002000:00010000:0.0:1789760739.424968:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21938176 -> 21942272 00010000:00010000:0.0:1789760739.424983:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.424995:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.425014:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.425021:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.425026:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.425034:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.425053:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.425061:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.425150:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515684096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31539us (31871us total) trans 4294972673 rc 0/0 00000100:00100000:0.0:1789760739.425160:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5385 00000100:00100000:0.0:1789760739.425170:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717818953775 time 717858884105 state 0x2 ref 0 00000100:00100000:2.0:1789760739.428382:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.428477:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684224 00000100:00100000:0.0:1789760739.428487:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684224 00000100:00100000:0.0:1789760739.428540:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717818953775 time 717858884105 state 0x0 ref 1 00000100:00100000:0.0:1789760739.428554:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717859144521 time 717858884105 state 0x1 ref 1 00000100:00100000:0.0:1789760739.428577:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717859144521 time 717893866783 state 0x0 ref 1 00000100:00100000:0.0:1789760739.428590:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515684224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.428640:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21942272, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.428653:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.428668:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760739.428673:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.428679:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.428703:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.463407:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972674 is committed 00002000:00010000:3.0:1789760739.463697:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21942272 -> 21946368 00010000:00010000:3.0:1789760739.463735:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.463749:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.463759:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.463766:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760739.463771:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.463780:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.463799:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.463813:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e4e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.463907:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515684224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35314us (35527us total) trans 4294972674 rc 0/0 00000100:00100000:3.0:1789760739.463932:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5386 00000100:00100000:3.0:1789760739.463943:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717859144521 time 717893866783 state 0x2 ref 0 00000100:00100000:0.0:1789760739.470225:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.470317:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684352 00000100:00100000:3.0:1789760739.470328:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684352 00000100:00100000:3.0:1789760739.470382:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717859144521 time 717893866783 state 0x0 ref 1 00000100:00100000:3.0:1789760739.470398:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717894127199 time 717893866783 state 0x1 ref 1 00000100:00100000:3.0:1789760739.470521:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717894127199 time 717935808658 state 0x0 ref 1 00000100:00100000:3.0:1789760739.470534:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.470665:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21946368, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.470675:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.470691:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760739.470695:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.470701:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.470756:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.504406:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972675 is committed 00002000:00010000:0.0:1789760739.504681:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21946368 -> 21950464 00010000:00010000:0.0:1789760739.504696:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.504788:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.504798:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.504805:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.504809:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.504818:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.504839:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.504846:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e4ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.505157:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34619us (34936us total) trans 4294972675 rc 0/0 00000100:00100000:1.0:1789760739.505169:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5387 00000100:00100000:1.0:1789760739.505179:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717894127199 time 717935808658 state 0x2 ref 0 00000100:00100000:3.0:1789760739.510835:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.510916:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684480 00000100:00100000:0.0:1789760739.510935:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684480 00000100:00100000:0.0:1789760739.510989:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717894127199 time 717935808658 state 0x0 ref 1 00000100:00100000:0.0:1789760739.511013:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717936069074 time 717935808658 state 0x1 ref 1 00000100:00100000:0.0:1789760739.511045:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717936069074 time 717976333795 state 0x0 ref 1 00000100:00100000:0.0:1789760739.511057:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.511103:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21950464, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.511113:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.511128:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760739.511133:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.511139:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.511152:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.540993:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21950464 -> 21954560 00010000:00010000:0.0:1789760739.541025:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.541043:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.541053:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.541060:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760739.541065:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.541074:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.541097:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.541104:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e4f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.541188:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515684480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30128us (30358us total) trans 4294972676 rc 0/0 00000100:00100000:0.0:1789760739.541198:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5388 00000100:00100000:0.0:1789760739.541207:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717936069074 time 717976333795 state 0x2 ref 0 00000001:00080000:0.0:1789760739.541850:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972676 is committed 00000100:00100000:2.0:1789760739.545523:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.545601:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684608 00000100:00100000:3.0:1789760739.545611:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684608 00000100:00100000:3.0:1789760739.545663:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717936069074 time 717976333795 state 0x0 ref 1 00000100:00100000:3.0:1789760739.545679:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717976594211 time 717976333795 state 0x1 ref 1 00000100:00100000:3.0:1789760739.545741:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717976594211 time 718011029360 state 0x0 ref 1 00000100:00100000:3.0:1789760739.545753:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.545799:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21954560, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.545807:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.545823:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.545827:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.545834:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.545848:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.570931:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972677 is committed 00002000:00010000:1.0:1789760739.571046:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21954560 -> 21958656 00010000:00010000:1.0:1789760739.571062:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.571074:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.571082:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.571093:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760739.571097:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.571105:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.571123:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.571129:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e4fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.571217:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25460us (25698us total) trans 4294972677 rc 0/0 00000100:00100000:1.0:1789760739.571227:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5389 00000100:00100000:1.0:1789760739.571237:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717976594211 time 718011029360 state 0x2 ref 0 00000100:00100000:0.0:1789760739.574238:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760739.575861:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684736 00000100:00100000:2.0:1789760739.575874:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684736 00000100:00100000:2.0:1789760739.576011:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 717976594211 time 718011029360 state 0x0 ref 1 00000100:00100000:2.0:1789760739.576031:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718011289776 time 718011029360 state 0x1 ref 1 00000100:00100000:2.0:1789760739.576098:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718011289776 time 718041385556 state 0x0 ref 1 00000100:00100000:2.0:1789760739.576112:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515684736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760739.576159:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21958656, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:2.0:1789760739.576168:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.576183:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.576188:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.576194:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.576208:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.625662:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972678 is committed 00002000:00010000:3.0:1789760739.626016:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21958656 -> 21962752 00010000:00010000:3.0:1789760739.626032:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.626047:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.626055:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.626062:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760739.626066:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.626075:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.626092:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.626100:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e501 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.626192:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515684736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50079us (51958us total) trans 4294972678 rc 0/0 00000100:00100000:3.0:1789760739.626205:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5390 00000100:00100000:3.0:1789760739.626215:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718011289776 time 718041385556 state 0x2 ref 0 00000100:00100000:3.0:1789760739.629519:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760739.629681:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684864 00000100:00100000:0.0:1789760739.629691:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684864 00000100:00100000:0.0:1789760739.629779:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718011289776 time 718041385556 state 0x0 ref 1 00000100:00100000:0.0:1789760739.629795:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718041645972 time 718041385556 state 0x1 ref 1 00000100:00100000:0.0:1789760739.629817:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718041645972 time 718095106351 state 0x0 ref 1 00000100:00100000:0.0:1789760739.629829:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760739.629875:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21962752, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:0.0:1789760739.629883:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.629898:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760739.629903:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.629910:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.629935:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.647322:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760739.657696:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972679 is committed 00002000:00010000:1.0:1789760739.657824:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21962752 -> 21966848 00010000:00010000:1.0:1789760739.657839:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.657852:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.657867:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.657874:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760739.657879:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.657888:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.657907:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.657922:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e508 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.658019:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515684864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28187us (28505us total) trans 4294972679 rc 0/0 00000100:00100000:1.0:1789760739.658029:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5391 00000100:00100000:1.0:1789760739.658039:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718041645972 time 718095106351 state 0x2 ref 0 00000100:00100000:0.0:1789760739.661564:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.661649:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515684992 00000100:00100000:3.0:1789760739.661658:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515684992 00000100:00100000:3.0:1789760739.661732:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718041645972 time 718095106351 state 0x0 ref 1 00000100:00100000:3.0:1789760739.661749:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718095366767 time 718095106351 state 0x1 ref 1 00000100:00100000:3.0:1789760739.661776:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718095366767 time 718127064842 state 0x0 ref 1 00000100:00100000:3.0:1789760739.661787:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515684992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.661831:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21966848, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.661840:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.661854:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.661858:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.661864:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.661877:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.699874:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972680 is committed 00002000:00010000:1.0:1789760739.700539:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21966848 -> 21970944 00010000:00010000:1.0:1789760739.700556:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.700576:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.700585:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.700593:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760739.700599:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.700609:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.700630:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.700638:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e50f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.700820:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515684992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39029us (39259us total) trans 4294972680 rc 0/0 00000100:00100000:1.0:1789760739.700830:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5392 00000100:00100000:1.0:1789760739.700839:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718095366767 time 718127064842 state 0x2 ref 0 00000100:00100000:0.0:1789760739.718130:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760739.718253:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685120 00000100:00100000:1.0:1789760739.718264:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685120 00000100:00100000:1.0:1789760739.718321:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718095366767 time 718127064842 state 0x0 ref 1 00000100:00100000:1.0:1789760739.718335:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718127325258 time 718127064842 state 0x1 ref 1 00000100:00100000:1.0:1789760739.718367:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718127325258 time 718183656441 state 0x0 ref 1 00000100:00100000:1.0:1789760739.718378:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760739.718425:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21970944, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:1.0:1789760739.718435:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.718450:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.718454:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.718461:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.718475:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.772738:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972681 is committed 00002000:00010000:1.0:1789760739.772811:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21970944 -> 21975040 00010000:00010000:1.0:1789760739.772826:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.772837:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.772851:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.772857:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760739.772862:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.772870:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.772888:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.772895:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e516 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.772986:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54605us (54862us total) trans 4294972681 rc 0/0 00000100:00100000:1.0:1789760739.773006:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5393 00000100:00100000:1.0:1789760739.773016:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718127325258 time 718183656441 state 0x2 ref 0 00000100:00100000:2.0:1789760739.780166:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760739.780249:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685248 00000100:00100000:1.0:1789760739.780260:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685248 00000100:00100000:1.0:1789760739.780314:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718127325258 time 718183656441 state 0x0 ref 1 00000100:00100000:1.0:1789760739.780331:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718183916857 time 718183656441 state 0x1 ref 1 00000100:00100000:1.0:1789760739.780365:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718183916857 time 718245654034 state 0x0 ref 1 00000100:00100000:1.0:1789760739.780377:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760739.780424:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21975040, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:1.0:1789760739.780433:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.780449:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.780453:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.780460:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.780473:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.819565:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972682 is committed 00002000:00010000:2.0:1789760739.819673:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21975040 -> 21979136 00010000:00010000:2.0:1789760739.819688:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.819699:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.819731:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.819738:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760739.819743:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.819752:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.819771:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.819780:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e51d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.819880:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39500us (39720us total) trans 4294972682 rc 0/0 00000100:00100000:2.0:1789760739.819891:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5394 00000100:00100000:2.0:1789760739.819901:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718183916857 time 718245654034 state 0x2 ref 0 00000100:00100000:1.0:1789760739.831140:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.832144:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685376 00000100:00100000:3.0:1789760739.832154:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685376 00000100:00100000:3.0:1789760739.832213:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718183916857 time 718245654034 state 0x0 ref 1 00000100:00100000:3.0:1789760739.832228:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718245914450 time 718245654034 state 0x1 ref 1 00000100:00100000:3.0:1789760739.832264:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718245914450 time 718297553118 state 0x0 ref 1 00000100:00100000:3.0:1789760739.832277:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.832327:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21979136, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa807 00010000:00010000:3.0:1789760739.832338:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.832354:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.832359:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.832365:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.832380:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.901811:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972683 is committed 00002000:00010000:3.0:1789760739.902112:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21979136 -> 21983232 00002000:00010000:3.0:1789760739.902117:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760739 -> 1789760740 00002000:00010000:3.0:1789760739.902119:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760739 -> 1789760740 00010000:00010000:3.0:1789760739.902134:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902147:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902156:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902163:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760739.902168:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902178:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902198:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902206:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e524 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.902303:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70025us (71168us total) trans 4294972683 rc 0/0 00000100:00100000:3.0:1789760739.902314:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5395 00000100:00100000:3.0:1789760739.902324:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718245914450 time 718297553118 state 0x2 ref 0 00000100:00100000:1.0:1789760739.905805:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.905847:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685504 00000100:00100000:3.0:1789760739.905856:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685504 00000100:00100000:3.0:1789760739.905899:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718245914450 time 718297553118 state 0x0 ref 1 00000100:00100000:3.0:1789760739.905910:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718297813534 time 718297553118 state 0x1 ref 1 00000100:00100000:3.0:1789760739.906088:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718297813534 time 718371375465 state 0x0 ref 1 00000100:00100000:3.0:1789760739.906099:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515685504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.906139:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21983232, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:3.0:1789760739.906145:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.906157:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.906161:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.906165:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.906175:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.945449:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21983232 -> 21987328 00010000:00010000:2.0:1789760739.945470:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.945491:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.945501:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.945508:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760739.945513:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.945522:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.945541:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.945547:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e52b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.945640:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515685504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39537us (39841us total) trans 4294972684 rc 0/0 00000100:00100000:2.0:1789760739.945650:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5396 00000100:00100000:2.0:1789760739.945660:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718297813534 time 718371375465 state 0x2 ref 0 00000001:00080000:0.0:1789760739.946171:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972684 is committed 00000100:00100000:0.0:1789760739.949360:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760739.949461:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685632 00000100:00100000:1.0:1789760739.949470:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685632 00000100:00100000:1.0:1789760739.949524:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718297813534 time 718371375465 state 0x0 ref 1 00000100:00100000:1.0:1789760739.949539:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718371635881 time 718371375465 state 0x1 ref 1 00000100:00100000:1.0:1789760739.949601:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718371635881 time 718414890040 state 0x0 ref 1 00000100:00100000:1.0:1789760739.949615:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760739.949665:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21987328, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:1.0:1789760739.949675:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.949690:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.949695:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.949701:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.949819:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760739.988280:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21987328 -> 21991424 00010000:00010000:1.0:1789760739.988299:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.988312:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.988321:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.988329:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760739.988333:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.988343:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.988362:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.988369:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e532 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.988528:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515685632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38910us (39172us total) trans 4294972685 rc 0/0 00000100:00100000:1.0:1789760739.988539:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5397 00000100:00100000:1.0:1789760739.988549:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718371635881 time 718414890040 state 0x2 ref 0 00000001:00080000:0.0:1789760739.989035:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972685 is committed 00000100:00100000:1.0:1789760739.992975:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760739.993063:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685760 00000100:00100000:3.0:1789760739.993073:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685760 00000100:00100000:3.0:1789760739.993125:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718371635881 time 718414890040 state 0x0 ref 1 00000100:00100000:3.0:1789760739.993139:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718415150456 time 718414890040 state 0x1 ref 1 00000100:00100000:3.0:1789760739.993176:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718415150456 time 718458465006 state 0x0 ref 1 00000100:00100000:3.0:1789760739.993188:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760739.993234:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21991424, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:3.0:1789760739.993243:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.993260:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.993264:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.993271:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.993285:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.020253:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21991424 -> 21995520 00010000:00010000:3.0:1789760740.020274:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.020286:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.020295:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.020302:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760740.020308:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.020316:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.020336:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.020343:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e539 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.020428:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515685760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27237us (27513us total) trans 4294972686 rc 0/0 00000100:00100000:3.0:1789760740.020438:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5398 00000100:00100000:3.0:1789760740.020449:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718415150456 time 718458465006 state 0x2 ref 0 00000001:00080000:3.0:1789760740.020792:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972686 is committed 00000100:00100000:2.0:1789760740.023546:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.023861:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515685888 00000100:00100000:1.0:1789760740.023870:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515685888 00000100:00080000:2.0:1789760740.023985:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760740.024039:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718415150456 time 718458465006 state 0x0 ref 1 00000100:00100000:1.0:1789760740.024055:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718458725422 time 718458465006 state 0x1 ref 1 00000100:00100000:1.0:1789760740.024085:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718458725422 time 718489373626 state 0x0 ref 1 00000100:00100000:1.0:1789760740.024098:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.024145:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21995520, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:1.0:1789760740.024154:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.024177:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760740.024181:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.024187:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.024200:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.052571:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972687 is committed 00002000:00010000:2.0:1789760740.052749:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21995520 -> 21999616 00010000:00010000:2.0:1789760740.052766:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.052777:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.052786:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.052792:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.052797:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.052805:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.052824:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.052832:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e540 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.052934:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515685888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28834us (29392us total) trans 4294972687 rc 0/0 00000100:00100000:2.0:1789760740.052944:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5399 00000100:00100000:2.0:1789760740.052954:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718458725422 time 718489373626 state 0x2 ref 0 00000100:00100000:3.0:1789760740.056876:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760740.056972:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686016 00000100:00100000:2.0:1789760740.056982:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686016 00000100:00100000:2.0:1789760740.057045:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718458725422 time 718489373626 state 0x0 ref 1 00000100:00100000:2.0:1789760740.057061:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718489634042 time 718489373626 state 0x1 ref 1 00000100:00100000:2.0:1789760740.057092:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718489634042 time 718522380474 state 0x0 ref 1 00000100:00100000:2.0:1789760740.057104:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515686016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760740.057148:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 21999616, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:2.0:1789760740.057157:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.057173:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760740.057177:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.057184:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.057197:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.089203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972688 is committed 00002000:00010000:0.0:1789760740.089305:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 21999616 -> 22003712 00010000:00010000:0.0:1789760740.089321:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.089334:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.089344:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.089352:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760740.089357:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.089366:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.089385:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.089395:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e547 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.089488:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515686016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32381us (32617us total) trans 4294972688 rc 0/0 00000100:00100000:0.0:1789760740.089503:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5400 00000100:00100000:0.0:1789760740.089514:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718489634042 time 718522380474 state 0x2 ref 0 00000100:00100000:0.0:1789760740.094097:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.094185:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686144 00000100:00100000:3.0:1789760740.094196:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686144 00000100:00100000:3.0:1789760740.094250:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718489634042 time 718522380474 state 0x0 ref 1 00000100:00100000:3.0:1789760740.094264:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718522640890 time 718522380474 state 0x1 ref 1 00000100:00100000:3.0:1789760740.094297:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718522640890 time 718559585953 state 0x0 ref 1 00000100:00100000:3.0:1789760740.094310:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515686144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.094355:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22003712, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:3.0:1789760740.094363:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.094378:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760740.094382:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.094389:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.094403:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.132467:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22003712 -> 22007808 00010000:00010000:0.0:1789760740.132489:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.132502:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.132512:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.132519:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760740.132524:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.132533:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.132552:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.132559:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e54e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.132671:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515686144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38358us (38580us total) trans 4294972689 rc 0/0 00000100:00100000:0.0:1789760740.132682:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5401 00000100:00100000:0.0:1789760740.132692:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718522640890 time 718559585953 state 0x2 ref 0 00000001:00080000:2.0:1789760740.133759:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972689 is committed 00000100:00100000:2.0:1789760740.135832:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.136028:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686272 00000100:00100000:3.0:1789760740.136039:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686272 00000100:00100000:3.0:1789760740.136090:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718522640890 time 718559585953 state 0x0 ref 1 00000100:00100000:3.0:1789760740.136106:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718559846369 time 718559585953 state 0x1 ref 1 00000100:00100000:3.0:1789760740.136139:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718559846369 time 718601427380 state 0x0 ref 1 00000100:00100000:3.0:1789760740.136152:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515686272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.136199:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22007808, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:3.0:1789760740.136208:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.136230:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760740.136235:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.136241:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.136256:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.186082:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972690 is committed 00002000:00010000:3.0:1789760740.186233:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22007808 -> 22011904 00010000:00010000:3.0:1789760740.186248:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.186260:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.186270:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.186278:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760740.186283:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.186293:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.186314:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.186325:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e555 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.186420:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515686272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50266us (50593us total) trans 4294972690 rc 0/0 00000100:00100000:3.0:1789760740.186430:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5402 00000100:00100000:3.0:1789760740.186440:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718559846369 time 718601427380 state 0x2 ref 0 00000100:00100000:3.0:1789760740.197742:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.197827:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686400 00000100:00100000:1.0:1789760740.197837:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686400 00000100:00100000:1.0:1789760740.197894:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718559846369 time 718601427380 state 0x0 ref 1 00000100:00100000:1.0:1789760740.197910:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718601687796 time 718601427380 state 0x1 ref 1 00000100:00100000:1.0:1789760740.197942:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718601687796 time 718663231775 state 0x0 ref 1 00000100:00100000:1.0:1789760740.197954:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515686400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.198015:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22011904, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:1.0:1789760740.198025:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.198040:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760740.198045:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.198051:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.198069:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.254961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972691 is committed 00002000:00010000:0.0:1789760740.255124:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22011904 -> 22016000 00010000:00010000:0.0:1789760740.255140:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.255151:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.255160:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.255167:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760740.255172:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.255182:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.255200:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.255214:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e55c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.255304:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515686400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57347us (57598us total) trans 4294972691 rc 0/0 00000100:00100000:0.0:1789760740.255315:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5403 00000100:00100000:0.0:1789760740.255326:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718601687796 time 718663231775 state 0x2 ref 0 00000100:00100000:0.0:1789760740.259944:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.260077:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686528 00000100:00100000:3.0:1789760740.260087:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686528 00000100:00100000:3.0:1789760740.260142:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718601687796 time 718663231775 state 0x0 ref 1 00000100:00100000:3.0:1789760740.260157:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718663492191 time 718663231775 state 0x1 ref 1 00000100:00100000:3.0:1789760740.260290:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718663492191 time 718725577521 state 0x0 ref 1 00000100:00100000:3.0:1789760740.260304:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515686528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.260365:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22016000, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:3.0:1789760740.260374:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.260390:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760740.260394:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.260401:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.260414:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.291323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972692 is committed 00002000:00010000:1.0:1789760740.291329:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22016000 -> 22020096 00010000:00010000:1.0:1789760740.291345:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.291358:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.291368:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.291375:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760740.291380:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.291389:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.291409:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.291417:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e563 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.291517:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515686528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31211us (31581us total) trans 4294972692 rc 0/0 00000100:00100000:1.0:1789760740.291528:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5404 00000100:00100000:1.0:1789760740.291537:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718663492191 time 718725577521 state 0x2 ref 0 00000100:00100000:2.0:1789760740.299570:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.299677:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686656 00000100:00100000:1.0:1789760740.299687:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686656 00000100:00100000:1.0:1789760740.299764:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718663492191 time 718725577521 state 0x0 ref 1 00000100:00100000:1.0:1789760740.299779:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718725837937 time 718725577521 state 0x1 ref 1 00000100:00100000:1.0:1789760740.299840:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718725837937 time 718765129408 state 0x0 ref 1 00000100:00100000:1.0:1789760740.299853:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515686656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760740.299932:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22020096, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa807 00010000:00010000:0.0:1789760740.299944:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.299960:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760740.299965:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.299971:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.299984:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.337750:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972693 is committed 00002000:00010000:2.0:1789760740.337839:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22020096 -> 22024192 00002000:00010000:2.0:1789760740.337844:0:13071:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:2.0:1789760740.337856:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.337867:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.337876:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.337883:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.337887:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.337896:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.337981:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.338001:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e56a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.338152:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515686656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38295us (38584us total) trans 4294972693 rc 0/0 00000100:00100000:2.0:1789760740.338164:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5405 00000100:00100000:2.0:1789760740.338182:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718725837937 time 718765129408 state 0x2 ref 0 00000100:00100000:3.0:1789760740.342660:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760740.345013:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686784 00000100:00100000:0.0:1789760740.345026:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686784 00000100:00100000:0.0:1789760740.345086:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718725837937 time 718765129408 state 0x0 ref 1 00000100:00100000:0.0:1789760740.345102:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718765389824 time 718765129408 state 0x1 ref 1 00000100:00100000:0.0:1789760740.345141:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718765389824 time 718810430108 state 0x0 ref 1 00000100:00100000:0.0:1789760740.345154:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515686784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760740.345199:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22024192, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:0.0:1789760740.345208:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.345223:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760740.345228:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.345235:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.345249:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.385135:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22024192 -> 22028288 00010000:00010000:0.0:1789760740.385153:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.385166:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.385175:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.385182:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760740.385188:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.385197:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.385216:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.385225:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e571 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.385308:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515686784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40152us (42653us total) trans 4294972694 rc 0/0 00000100:00100000:0.0:1789760740.385319:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5406 00000100:00100000:0.0:1789760740.385329:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718765389824 time 718810430108 state 0x2 ref 0 00000001:00080000:2.0:1789760740.385834:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972694 is committed 00000100:00100000:0.0:1789760740.388869:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.389030:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515686912 00000100:00100000:3.0:1789760740.389039:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515686912 00000100:00100000:3.0:1789760740.389088:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718765389824 time 718810430108 state 0x0 ref 1 00000100:00100000:3.0:1789760740.389103:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718810690524 time 718810430108 state 0x1 ref 1 00000100:00100000:3.0:1789760740.389133:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718810690524 time 718854422156 state 0x0 ref 1 00000100:00100000:3.0:1789760740.389144:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515686912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.389189:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22028288, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:3.0:1789760740.389198:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.389212:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760740.389217:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.389224:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.389237:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.420184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972695 is committed 00002000:00010000:0.0:1789760740.420282:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22028288 -> 22032384 00010000:00010000:0.0:1789760740.420297:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.420310:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.420320:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.420333:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760740.420339:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.420349:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.420367:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.420375:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e578 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.420570:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515686912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31315us (31598us total) trans 4294972695 rc 0/0 00000100:00100000:0.0:1789760740.420582:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5407 00000100:00100000:0.0:1789760740.420592:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718810690524 time 718854422156 state 0x2 ref 0 00000100:00100000:2.0:1789760740.423844:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.423948:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687040 00000100:00100000:3.0:1789760740.423958:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687040 00000100:00100000:3.0:1789760740.424019:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718810690524 time 718854422156 state 0x0 ref 1 00000100:00100000:3.0:1789760740.424043:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718854682572 time 718854422156 state 0x1 ref 1 00000100:00100000:3.0:1789760740.424073:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718854682572 time 718889361517 state 0x0 ref 1 00000100:00100000:3.0:1789760740.424086:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515687040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.424132:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22032384, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:3.0:1789760740.424142:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.424159:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760740.424164:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.424171:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.424185:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.459271:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22032384 -> 22036480 00010000:00010000:1.0:1789760740.459289:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.459300:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.459305:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.459313:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760740.459317:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.459322:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.459337:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.459344:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e57f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.459432:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515687040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35343us (35592us total) trans 4294972696 rc 0/0 00000100:00100000:1.0:1789760740.459441:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5408 00000100:00100000:1.0:1789760740.459451:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718854682572 time 718889361517 state 0x2 ref 0 00000001:00080000:2.0:1789760740.459522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972696 is committed 00000100:00100000:3.0:1789760740.463222:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.463303:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687168 00000100:00100000:1.0:1789760740.463313:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687168 00000100:00100000:1.0:1789760740.463369:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718854682572 time 718889361517 state 0x0 ref 1 00000100:00100000:1.0:1789760740.463385:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718889621933 time 718889361517 state 0x1 ref 1 00000100:00100000:1.0:1789760740.463414:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718889621933 time 718928703504 state 0x0 ref 1 00000100:00100000:1.0:1789760740.463430:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.463476:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22036480, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:1.0:1789760740.463485:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.463511:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760740.463516:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.463523:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.463538:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.501596:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972697 is committed 00002000:00010000:1.0:1789760740.502170:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22036480 -> 22040576 00010000:00010000:1.0:1789760740.502490:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.502505:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.502514:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.502521:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760740.502526:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.502536:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.502556:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.502563:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e586 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.502653:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39220us (39436us total) trans 4294972697 rc 0/0 00000100:00100000:1.0:1789760740.502663:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5409 00000100:00100000:1.0:1789760740.502673:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718889621933 time 718928703504 state 0x2 ref 0 00000100:00100000:0.0:1789760740.506284:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.506419:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687296 00000100:00100000:1.0:1789760740.506429:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687296 00000100:00100000:1.0:1789760740.506480:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718889621933 time 718928703504 state 0x0 ref 1 00000100:00100000:1.0:1789760740.506496:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718928963920 time 718928703504 state 0x1 ref 1 00000100:00100000:1.0:1789760740.506524:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718928963920 time 718971813376 state 0x0 ref 1 00000100:00100000:1.0:1789760740.506537:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.506583:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22040576, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:1.0:1789760740.506592:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.506606:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760740.506611:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.506619:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.506633:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760740.545446:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22040576 -> 22044672 00010000:00010000:2.0:1789760740.545465:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.545478:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.545484:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972698 is committed 00010000:00010000:2.0:1789760740.545488:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.545497:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.545502:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.545511:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.545543:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.545551:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e58d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.545637:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39098us (39358us total) trans 4294972698 rc 0/0 00000100:00100000:2.0:1789760740.545648:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5410 00000100:00100000:2.0:1789760740.545657:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718928963920 time 718971813376 state 0x2 ref 0 00000100:00100000:2.0:1789760740.556595:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760740.556685:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687424 00000100:00100000:0.0:1789760740.556695:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687424 00000100:00100000:0.0:1789760740.556770:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718928963920 time 718971813376 state 0x0 ref 1 00000100:00100000:0.0:1789760740.556786:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718972073792 time 718971813376 state 0x1 ref 1 00000100:00100000:1.0:1789760740.556886:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718972073792 time 719022173141 state 0x0 ref 1 00000100:00100000:1.0:1789760740.557000:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.557047:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22044672, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:1.0:1789760740.557055:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.557070:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760740.557074:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.557080:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.557093:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.591831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972699 is committed 00002000:00010000:1.0:1789760740.591969:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22044672 -> 22048768 00010000:00010000:1.0:1789760740.591985:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.592006:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.592016:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.592024:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760740.592028:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.592037:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.592057:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.592064:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e594 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.592152:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35249us (35561us total) trans 4294972699 rc 0/0 00000100:00100000:1.0:1789760740.592163:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5411 00000100:00100000:1.0:1789760740.592173:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718972073792 time 719022173141 state 0x2 ref 0 00000100:00100000:0.0:1789760740.598082:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760740.598170:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687552 00000100:00100000:2.0:1789760740.598179:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687552 00000100:00100000:2.0:1789760740.598230:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 718972073792 time 719022173141 state 0x0 ref 1 00000100:00100000:2.0:1789760740.598246:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719022433557 time 719022173141 state 0x1 ref 1 00000100:00100000:2.0:1789760740.598306:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719022433557 time 719063594635 state 0x0 ref 1 00000100:00100000:2.0:1789760740.598319:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760740.598365:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22048768, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:2.0:1789760740.598374:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.598389:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.598393:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.598400:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.598414:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.631207:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972700 is committed 00002000:00010000:3.0:1789760740.631325:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22048768 -> 22052864 00010000:00010000:3.0:1789760740.631342:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.631354:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.631363:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.631369:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760740.631374:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.631382:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.631400:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.631407:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1e59b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.631509:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33188us (33432us total) trans 4294972700 rc 0/0 00000100:00100000:3.0:1789760740.631519:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5412 00000100:00100000:3.0:1789760740.631528:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719022433557 time 719063594635 state 0x2 ref 0 00000100:00100000:0.0:1789760740.640096:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.640194:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687680 00000100:00100000:3.0:1789760740.640225:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687680 00000100:00100000:3.0:1789760740.640309:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719022433557 time 719063594635 state 0x0 ref 1 00000100:00100000:3.0:1789760740.640330:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719063855051 time 719063594635 state 0x1 ref 1 00000100:00100000:3.0:1789760740.640365:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719063855051 time 719105654226 state 0x0 ref 1 00000100:00100000:3.0:1789760740.640382:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.640454:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22052864, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:3.0:1789760740.640466:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.640486:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.640496:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.640505:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.640523:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.669874:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760740.683753:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972701 is committed 00002000:00010000:3.0:1789760740.685541:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22052864 -> 22056960 00010000:00010000:3.0:1789760740.685564:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.685577:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.685587:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.685594:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760740.685598:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.685607:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.685696:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.685720:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.685865:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45483us (45788us total) trans 4294972701 rc 0/0 00000100:00100000:3.0:1789760740.685875:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5413 00000100:00100000:3.0:1789760740.685885:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719063855051 time 719105654226 state 0x2 ref 0 00000100:00100000:0.0:1789760740.690874:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.691020:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687808 00000100:00100000:1.0:1789760740.691030:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687808 00000100:00100000:1.0:1789760740.691085:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719063855051 time 719105654226 state 0x0 ref 1 00000100:00100000:1.0:1789760740.691099:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719105914642 time 719105654226 state 0x1 ref 1 00000100:00100000:1.0:1789760740.691173:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719105914642 time 719156460173 state 0x0 ref 1 00000100:00100000:1.0:1789760740.691186:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.691234:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22056960, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:1.0:1789760740.691242:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.691256:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760740.691260:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.691266:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.691280:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.732620:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972702 is committed 00002000:00010000:2.0:1789760740.732803:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22056960 -> 22061056 00010000:00010000:2.0:1789760740.732818:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.732831:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.732840:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.732846:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.732851:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.732860:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.732879:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.732887:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e5a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.733079:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515687808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41891us (42208us total) trans 4294972702 rc 0/0 00000100:00100000:2.0:1789760740.733094:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5414 00000100:00100000:2.0:1789760740.733104:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719105914642 time 719156460173 state 0x2 ref 0 00000100:00100000:0.0:1789760740.736379:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760740.736459:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515687936 00000100:00100000:1.0:1789760740.736468:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515687936 00000100:00100000:1.0:1789760740.736510:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719105914642 time 719156460173 state 0x0 ref 1 00000100:00100000:1.0:1789760740.736524:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719156720589 time 719156460173 state 0x1 ref 1 00000100:00100000:1.0:1789760740.736551:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719156720589 time 719201840183 state 0x0 ref 1 00000100:00100000:1.0:1789760740.736566:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515687936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760740.736603:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22061056, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:1.0:1789760740.736610:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.736621:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760740.736624:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.736629:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.736638:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.770290:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972703 is committed 00002000:00010000:3.0:1789760740.770350:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22061056 -> 22065152 00010000:00010000:3.0:1789760740.770362:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.770374:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.770380:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.770384:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760740.770387:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.770393:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.770406:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.770411:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e5b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.770511:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515687936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33938us (34133us total) trans 4294972703 rc 0/0 00000100:00100000:3.0:1789760740.770521:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5415 00000100:00100000:3.0:1789760740.770530:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719156720589 time 719201840183 state 0x2 ref 0 00000100:00100000:0.0:1789760740.773956:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760740.774055:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688064 00000100:00100000:2.0:1789760740.774065:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688064 00000100:00100000:2.0:1789760740.774113:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719156720589 time 719201840183 state 0x0 ref 1 00000100:00100000:2.0:1789760740.774127:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719202100599 time 719201840183 state 0x1 ref 1 00000100:00100000:3.0:1789760740.774207:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719202100599 time 719239493073 state 0x0 ref 1 00000100:00100000:3.0:1789760740.774224:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.774279:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22065152, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:3.0:1789760740.774288:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.774303:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.774308:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.774314:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.774328:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.806607:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972704 is committed 00002000:00010000:3.0:1789760740.807033:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22065152 -> 22069248 00010000:00010000:3.0:1789760740.807048:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.807061:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.807069:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.807076:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760740.807081:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.807090:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.807186:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.807194:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.807271:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33048us (33321us total) trans 4294972704 rc 0/0 00000100:00100000:3.0:1789760740.807280:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5416 00000100:00100000:3.0:1789760740.807289:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719202100599 time 719239493073 state 0x2 ref 0 00000100:00100000:0.0:1789760740.812252:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.812505:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688192 00000100:00100000:3.0:1789760740.812522:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688192 00000100:00100000:3.0:1789760740.812597:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719202100599 time 719239493073 state 0x0 ref 1 00000100:00100000:3.0:1789760740.812616:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719239753489 time 719239493073 state 0x1 ref 1 00000100:00100000:3.0:1789760740.812900:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719239753489 time 719278186549 state 0x0 ref 1 00000100:00100000:3.0:1789760740.813053:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.813126:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22069248, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb007 00010000:00010000:3.0:1789760740.813139:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.813167:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.813172:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.813180:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.813198:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.847208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972705 is committed 00002000:00010000:2.0:1789760740.847318:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22069248 -> 22073344 00002000:00010000:2.0:1789760740.847323:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760740 -> 1789760741 00002000:00010000:2.0:1789760740.847326:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760740 -> 1789760741 00010000:00010000:2.0:1789760740.847338:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.847349:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.847357:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.847364:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.847368:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.847376:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.847395:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.847402:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e5be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.847501:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34449us (35255us total) trans 4294972705 rc 0/0 00000100:00100000:2.0:1789760740.847510:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5417 00000100:00100000:2.0:1789760740.847520:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719239753489 time 719278186549 state 0x2 ref 0 00000100:00100000:0.0:1789760740.852763:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.852842:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688320 00000100:00100000:3.0:1789760740.852851:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688320 00000100:00100000:3.0:1789760740.852906:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719239753489 time 719278186549 state 0x0 ref 1 00000100:00100000:3.0:1789760740.852921:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719278446965 time 719278186549 state 0x1 ref 1 00000100:00100000:3.0:1789760740.852951:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719278446965 time 719318240813 state 0x0 ref 1 00000100:00100000:3.0:1789760740.852961:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515688320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.853014:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22073344, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760740.853022:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.853034:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.853037:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.853048:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.853064:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.879106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972706 is committed 00002000:00010000:2.0:1789760740.879191:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22073344 -> 22077440 00010000:00010000:2.0:1789760740.879204:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.879213:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.879219:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.879223:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.879226:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.879236:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.879251:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.879255:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e5c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.879448:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515688320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26483us (26740us total) trans 4294972706 rc 0/0 00000100:00100000:2.0:1789760740.879458:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5418 00000100:00100000:2.0:1789760740.879468:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719278446965 time 719318240813 state 0x2 ref 0 00000100:00100000:1.0:1789760740.883067:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760740.883115:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688448 00000100:00100000:2.0:1789760740.883125:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688448 00000100:00100000:2.0:1789760740.883177:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719278446965 time 719318240813 state 0x0 ref 1 00000100:00100000:2.0:1789760740.883191:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719318501229 time 719318240813 state 0x1 ref 1 00000100:00100000:2.0:1789760740.883268:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719318501229 time 719348556071 state 0x0 ref 1 00000100:00100000:2.0:1789760740.883282:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760740.883340:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22077440, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760740.883348:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.883365:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.883370:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.883376:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.883389:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.914390:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972707 is committed 00002000:00010000:2.0:1789760740.914491:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22077440 -> 22081536 00010000:00010000:2.0:1789760740.914506:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.914518:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.914528:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.914535:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.914540:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.914550:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.914569:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.914578:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.914665:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31380us (31603us total) trans 4294972707 rc 0/0 00000100:00100000:2.0:1789760740.914675:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5419 00000100:00100000:2.0:1789760740.914684:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719318501229 time 719348556071 state 0x2 ref 0 00000100:00100000:0.0:1789760740.927229:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760740.927330:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688576 00000100:00100000:2.0:1789760740.927341:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688576 00000100:00100000:2.0:1789760740.927392:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719318501229 time 719348556071 state 0x0 ref 1 00000100:00100000:2.0:1789760740.927406:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719348816487 time 719348556071 state 0x1 ref 1 00000100:00100000:2.0:1789760740.927455:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719348816487 time 719392743435 state 0x0 ref 1 00000100:00100000:2.0:1789760740.927467:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760740.927515:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22081536, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760740.927524:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.927540:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.927544:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.927550:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.927571:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.958976:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972708 is committed 00002000:00010000:2.0:1789760740.959137:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22081536 -> 22085632 00010000:00010000:2.0:1789760740.959152:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.959165:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.959174:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.959182:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760740.959188:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.959198:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.959227:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.959236:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e5d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.959332:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31861us (32106us total) trans 4294972708 rc 0/0 00000100:00100000:2.0:1789760740.959343:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5420 00000100:00100000:2.0:1789760740.959352:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719348816487 time 719392743435 state 0x2 ref 0 00000100:00100000:1.0:1789760740.962896:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.965118:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688704 00000100:00100000:3.0:1789760740.965133:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688704 00000100:00100000:3.0:1789760740.965194:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719348816487 time 719392743435 state 0x0 ref 1 00000100:00100000:3.0:1789760740.965210:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719393003851 time 719392743435 state 0x1 ref 1 00000100:00100000:2.0:1789760740.965375:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719393003851 time 719430661145 state 0x0 ref 1 00000100:00100000:2.0:1789760740.965392:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760740.965444:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22085632, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760740.965458:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.965473:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.965478:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.965484:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.965497:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.995198:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972709 is committed 00002000:00010000:0.0:1789760740.995410:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22085632 -> 22089728 00010000:00010000:0.0:1789760740.995422:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.995432:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.995438:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.995442:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760740.995446:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.995452:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.995469:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.995474:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1e5da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.995547:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515688704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30157us (32657us total) trans 4294972709 rc 0/0 00000100:00100000:0.0:1789760740.995554:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5421 00000100:00100000:0.0:1789760740.995562:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719393003851 time 719430661145 state 0x2 ref 0 00000100:00100000:0.0:1789760740.999451:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760740.999652:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688832 00000100:00100000:3.0:1789760740.999662:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688832 00000100:00100000:3.0:1789760740.999737:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719393003851 time 719430661145 state 0x0 ref 1 00000100:00100000:3.0:1789760740.999752:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719430921561 time 719430661145 state 0x1 ref 1 00000100:00100000:3.0:1789760740.999830:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719430921561 time 719465118002 state 0x0 ref 1 00000100:00100000:3.0:1789760740.999844:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515688832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760740.999890:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22089728, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760740.999899:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.999925:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.999929:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.999934:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.999947:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.039201:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972710 is committed 00002000:00010000:3.0:1789760741.039232:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22089728 -> 22093824 00010000:00010000:3.0:1789760741.039247:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.039259:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.039268:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.039276:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760741.039281:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.039290:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.039310:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.039318:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.039466:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515688832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39620us (40020us total) trans 4294972710 rc 0/0 00000100:00100000:3.0:1789760741.039476:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5422 00000100:00100000:3.0:1789760741.039585:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719430921561 time 719465118002 state 0x2 ref 0 00000100:00100000:2.0:1789760741.047103:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760741.047200:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515688960 00000100:00100000:3.0:1789760741.047210:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515688960 00000100:00100000:3.0:1789760741.047271:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719430921561 time 719465118002 state 0x0 ref 1 00000100:00100000:3.0:1789760741.047287:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719465378418 time 719465118002 state 0x1 ref 1 00000100:00100000:3.0:1789760741.047314:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719465378418 time 719512603084 state 0x0 ref 1 00000100:00100000:3.0:1789760741.047326:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515688960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760741.047386:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22093824, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760741.047395:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.047411:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760741.047416:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.047422:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.047435:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.076510:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972711 is committed 00002000:00010000:3.0:1789760741.076683:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22093824 -> 22097920 00010000:00010000:3.0:1789760741.076698:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.076741:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.076751:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.076758:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760741.076763:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.076772:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.076793:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.076801:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e5e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.076881:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515688960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29552us (29784us total) trans 4294972711 rc 0/0 00000100:00100000:3.0:1789760741.076893:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5423 00000100:00100000:3.0:1789760741.077362:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719465378418 time 719512603084 state 0x2 ref 0 00000100:00100000:1.0:1789760741.080205:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760741.080346:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689088 00000100:00100000:3.0:1789760741.080357:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689088 00000100:00100000:3.0:1789760741.080404:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719465378418 time 719512603084 state 0x0 ref 1 00000100:00100000:3.0:1789760741.080418:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719512863500 time 719512603084 state 0x1 ref 1 00000100:00100000:3.0:1789760741.080450:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719512863500 time 719545739421 state 0x0 ref 1 00000100:00100000:3.0:1789760741.080462:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515689088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760741.080508:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22097920, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760741.080517:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.080533:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760741.080537:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.080544:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.080558:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.109352:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972712 is committed 00002000:00010000:3.0:1789760741.109437:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22097920 -> 22102016 00010000:00010000:3.0:1789760741.109452:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.109466:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.109474:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.109481:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760741.109486:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.109495:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.109519:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.109526:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e5ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.109660:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515689088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29194us (29459us total) trans 4294972712 rc 0/0 00000100:00100000:3.0:1789760741.109671:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5424 00000100:00100000:3.0:1789760741.109681:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719512863500 time 719545739421 state 0x2 ref 0 00000100:00100000:0.0:1789760741.115607:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760741.115701:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689216 00000100:00100000:2.0:1789760741.115737:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689216 00000100:00100000:2.0:1789760741.115792:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719512863500 time 719545739421 state 0x0 ref 1 00000100:00100000:2.0:1789760741.115810:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719545999837 time 719545739421 state 0x1 ref 1 00000100:00100000:2.0:1789760741.115846:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719545999837 time 719581134954 state 0x0 ref 1 00000100:00100000:2.0:1789760741.115860:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760741.115911:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22102016, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760741.115921:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.115937:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760741.115941:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.115947:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.115960:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.144634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972713 is committed 00002000:00010000:0.0:1789760741.144766:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22102016 -> 22106112 00010000:00010000:0.0:1789760741.144786:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.144798:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.144807:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.144815:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.144819:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.144828:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.144852:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.144861:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e5f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.144961:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29100us (29358us total) trans 4294972713 rc 0/0 00000100:00100000:0.0:1789760741.144971:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5425 00000100:00100000:0.0:1789760741.144980:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719545999837 time 719581134954 state 0x2 ref 0 00000100:00100000:2.0:1789760741.148550:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.148654:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689344 00000100:00100000:0.0:1789760741.148665:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689344 00000100:00100000:0.0:1789760741.148744:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719545999837 time 719581134954 state 0x0 ref 1 00000100:00100000:0.0:1789760741.148760:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719581395370 time 719581134954 state 0x1 ref 1 00000100:00100000:0.0:1789760741.148790:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719581395370 time 719614079386 state 0x0 ref 1 00000100:00100000:0.0:1789760741.148802:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515689344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760741.148854:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22106112, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:0.0:1789760741.148863:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.148879:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760741.148884:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.148891:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.148910:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.190271:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972714 is committed 00002000:00010000:0.0:1789760741.190432:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22106112 -> 22110208 00010000:00010000:0.0:1789760741.190449:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.190463:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.190473:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.190481:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.190485:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.190495:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.190520:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.190529:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e5fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.190622:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515689344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41818us (42078us total) trans 4294972714 rc 0/0 00000100:00100000:0.0:1789760741.192385:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5426 00000100:00100000:0.0:1789760741.192395:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719581395370 time 719614079386 state 0x2 ref 0 00000100:00100000:3.0:1789760741.193781:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.193862:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689472 00000100:00100000:0.0:1789760741.193872:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689472 00000100:00100000:0.0:1789760741.193930:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719581395370 time 719614079386 state 0x0 ref 1 00000100:00100000:0.0:1789760741.193945:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719614339802 time 719614079386 state 0x1 ref 1 00000100:00100000:0.0:1789760741.193972:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719614339802 time 719659260896 state 0x0 ref 1 00000100:00100000:0.0:1789760741.193994:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760741.194048:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22110208, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:0.0:1789760741.194057:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.194072:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760741.194076:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.194083:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.194097:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.236203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972715 is committed 00002000:00010000:0.0:1789760741.236282:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22110208 -> 22114304 00010000:00010000:0.0:1789760741.236297:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236309:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236318:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236326:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.236331:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236340:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236358:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.236370:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e604 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.236453:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42465us (42673us total) trans 4294972715 rc 0/0 00000100:00100000:0.0:1789760741.236463:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5427 00000100:00100000:0.0:1789760741.236473:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719614339802 time 719659260896 state 0x2 ref 0 00000100:00100000:1.0:1789760741.240806:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760741.240908:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689600 00000100:00100000:3.0:1789760741.240919:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689600 00000100:00100000:3.0:1789760741.240976:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719614339802 time 719659260896 state 0x0 ref 1 00000100:00100000:3.0:1789760741.241159:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719659521312 time 719659260896 state 0x1 ref 1 00000100:00100000:3.0:1789760741.241229:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719659521312 time 719706516801 state 0x0 ref 1 00000100:00100000:3.0:1789760741.241243:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515689600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760741.241291:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22114304, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760741.241301:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.241317:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760741.241321:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.241328:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.241342:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.292200:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972716 is committed 00002000:00010000:3.0:1789760741.292352:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22114304 -> 22118400 00010000:00010000:3.0:1789760741.292367:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.292387:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.292395:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.292402:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760741.292407:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.292416:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.292433:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.292441:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e60b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.292529:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515689600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51285us (51728us total) trans 4294972716 rc 0/0 00000100:00100000:3.0:1789760741.292538:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5428 00000100:00100000:3.0:1789760741.292547:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719659521312 time 719706516801 state 0x2 ref 0 00000100:00100000:0.0:1789760741.299382:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760741.299460:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689728 00000100:00100000:3.0:1789760741.299471:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689728 00000100:00100000:3.0:1789760741.299525:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719659521312 time 719706516801 state 0x0 ref 1 00000100:00100000:3.0:1789760741.299541:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719706777217 time 719706516801 state 0x1 ref 1 00000100:00100000:3.0:1789760741.299570:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719706777217 time 719764859170 state 0x0 ref 1 00000100:00100000:3.0:1789760741.299582:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760741.299637:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22118400, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760741.299647:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.299663:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760741.299668:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.299675:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.299689:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.340483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972717 is committed 00002000:00010000:1.0:1789760741.340587:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22118400 -> 22122496 00010000:00010000:1.0:1789760741.340603:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.340623:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.340632:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.340640:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760741.340645:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.340653:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.340671:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.340679:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e612 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.340891:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41306us (41514us total) trans 4294972717 rc 0/0 00000100:00100000:1.0:1789760741.341412:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5429 00000100:00100000:1.0:1789760741.341425:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719706777217 time 719764859170 state 0x2 ref 0 00000100:00100000:0.0:1789760741.348406:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760741.348495:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689856 00000100:00100000:3.0:1789760741.348506:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689856 00000100:00100000:3.0:1789760741.348558:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719706777217 time 719764859170 state 0x0 ref 1 00000100:00100000:3.0:1789760741.348572:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719765119586 time 719764859170 state 0x1 ref 1 00000100:00100000:3.0:1789760741.348602:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719765119586 time 719813891370 state 0x0 ref 1 00000100:00100000:3.0:1789760741.348618:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760741.348667:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22122496, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760741.348677:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.348694:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760741.348698:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.348705:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.348755:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.385856:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972718 is committed 00002000:00010000:1.0:1789760741.386096:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22122496 -> 22126592 00010000:00010000:1.0:1789760741.386111:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.386124:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.386133:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.386141:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760741.386146:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.386156:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.386176:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.386184:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e619 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.386308:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37687us (37908us total) trans 4294972718 rc 0/0 00000100:00100000:1.0:1789760741.386319:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5430 00000100:00100000:1.0:1789760741.386329:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719765119586 time 719813891370 state 0x2 ref 0 00000100:00100000:0.0:1789760741.392953:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760741.393082:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515689984 00000100:00100000:2.0:1789760741.393092:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515689984 00000100:00100000:2.0:1789760741.393146:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719765119586 time 719813891370 state 0x0 ref 1 00000100:00100000:2.0:1789760741.393161:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719814151786 time 719813891370 state 0x1 ref 1 00000100:00100000:2.0:1789760741.393194:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719814151786 time 719858483353 state 0x0 ref 1 00000100:00100000:2.0:1789760741.393209:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760741.393257:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22126592, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760741.393267:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.393282:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760741.393287:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.393292:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.393305:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.441337:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972719 is committed 00002000:00010000:0.0:1789760741.441411:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22126592 -> 22130688 00010000:00010000:0.0:1789760741.441426:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.441439:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.441447:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.441453:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.441459:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.441467:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.441484:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.441492:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e620 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.441689:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515689984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48371us (48634us total) trans 4294972719 rc 0/0 00000100:00100000:0.0:1789760741.441701:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5431 00000100:00100000:0.0:1789760741.441736:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719814151786 time 719858483353 state 0x2 ref 0 00000100:00100000:0.0:1789760741.449542:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760741.449604:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690112 00000100:00100000:2.0:1789760741.449613:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690112 00000100:00100000:2.0:1789760741.449667:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719814151786 time 719858483353 state 0x0 ref 1 00000100:00100000:2.0:1789760741.449682:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719858743769 time 719858483353 state 0x1 ref 1 00000100:00100000:2.0:1789760741.449795:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719858743769 time 719915083247 state 0x0 ref 1 00000100:00100000:2.0:1789760741.449808:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760741.449855:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22130688, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760741.449864:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.449883:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760741.449887:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.449894:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.449918:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.480821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972720 is committed 00002000:00010000:0.0:1789760741.481133:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22130688 -> 22134784 00010000:00010000:0.0:1789760741.481149:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.481162:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.481173:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.481187:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.481192:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.481202:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.481287:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.481297:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e627 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.481419:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31609us (31884us total) trans 4294972720 rc 0/0 00000100:00100000:0.0:1789760741.481430:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5432 00000100:00100000:0.0:1789760741.481440:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719858743769 time 719915083247 state 0x2 ref 0 00000100:00100000:2.0:1789760741.486869:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.487018:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690240 00000100:00100000:0.0:1789760741.487027:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690240 00000100:00100000:0.0:1789760741.487139:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719858743769 time 719915083247 state 0x0 ref 1 00000100:00100000:0.0:1789760741.487154:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719915343663 time 719915083247 state 0x1 ref 1 00000100:00100000:1.0:1789760741.488040:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719915343663 time 719953323484 state 0x0 ref 1 00000100:00100000:1.0:1789760741.488059:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760741.488110:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22134784, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:1.0:1789760741.488119:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.488134:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760741.488138:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.488145:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.488159:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.520418:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972721 is committed 00002000:00010000:1.0:1789760741.520477:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22134784 -> 22138880 00010000:00010000:1.0:1789760741.520492:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.520505:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.520514:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.520527:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760741.520532:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.520541:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.520560:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.520567:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e62e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.520649:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32592us (33785us total) trans 4294972721 rc 0/0 00000100:00100000:1.0:1789760741.520659:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5433 00000100:00100000:1.0:1789760741.520669:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719915343663 time 719953323484 state 0x2 ref 0 00000100:00100000:1.0:1789760741.528065:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760741.528410:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690368 00000100:00100000:3.0:1789760741.528429:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690368 00000100:00100000:3.0:1789760741.528507:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719915343663 time 719953323484 state 0x0 ref 1 00000100:00100000:3.0:1789760741.528527:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719953583900 time 719953323484 state 0x1 ref 1 00000100:00100000:3.0:1789760741.528630:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719953583900 time 719993916872 state 0x0 ref 1 00000100:00100000:3.0:1789760741.528646:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760741.528764:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22138880, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:3.0:1789760741.528776:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.528802:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760741.528807:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.528815:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.528831:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760741.566284:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22138880 -> 22142976 00010000:00010000:0.0:1789760741.566312:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.566330:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.566341:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.566348:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.566354:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.566365:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.566395:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.566404:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1e635 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.566565:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515690368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37915us (38561us total) trans 4294972722 rc 0/0 00000100:00100000:0.0:1789760741.566579:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5434 00000100:00100000:0.0:1789760741.566593:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719953583900 time 719993916872 state 0x2 ref 0 00000001:00080000:1.0:1789760741.566762:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972722 is committed 00000100:00100000:3.0:1789760741.570287:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760741.570378:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690496 00000100:00100000:2.0:1789760741.570391:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690496 00000100:00100000:2.0:1789760741.570465:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719953583900 time 719993916872 state 0x0 ref 1 00000100:00100000:2.0:1789760741.570485:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719994177288 time 719993916872 state 0x1 ref 1 00000100:00100000:2.0:1789760741.570519:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719994177288 time 720035808160 state 0x0 ref 1 00000100:00100000:2.0:1789760741.570534:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760741.570598:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22142976, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760741.570609:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.570642:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.570648:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.570656:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.570673:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.603699:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972723 is committed 00002000:00010000:2.0:1789760741.604073:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22142976 -> 22147072 00010000:00010000:2.0:1789760741.604095:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.604110:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.604120:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.604128:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760741.604134:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.604143:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.604168:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.604176:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e63c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.604307:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33768us (34026us total) trans 4294972723 rc 0/0 00000100:00100000:2.0:1789760741.604321:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5435 00000100:00100000:2.0:1789760741.604333:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719994177288 time 720035808160 state 0x2 ref 0 00000100:00100000:1.0:1789760741.608147:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.608272:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690624 00000100:00100000:0.0:1789760741.608287:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690624 00000100:00100000:0.0:1789760741.608360:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 719994177288 time 720035808160 state 0x0 ref 1 00000100:00100000:0.0:1789760741.608380:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720036068576 time 720035808160 state 0x1 ref 1 00000100:00100000:0.0:1789760741.608455:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720036068576 time 720073743006 state 0x0 ref 1 00000100:00100000:0.0:1789760741.608470:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515690624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760741.608537:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22147072, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:0.0:1789760741.608548:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.608569:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.608574:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.608582:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.608599:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.638031:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972724 is committed 00002000:00010000:0.0:1789760741.638121:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22147072 -> 22151168 00010000:00010000:0.0:1789760741.638141:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.638157:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.638174:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.638183:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.638188:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.638198:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.638221:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.638229:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e643 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.638380:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515690624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29905us (30242us total) trans 4294972724 rc 0/0 00000100:00100000:0.0:1789760741.638392:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5436 00000100:00100000:0.0:1789760741.638405:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720036068576 time 720073743006 state 0x2 ref 0 00000100:00100000:1.0:1789760741.641488:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.641635:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690752 00000100:00100000:0.0:1789760741.641648:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690752 00000100:00100000:0.0:1789760741.641740:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720036068576 time 720073743006 state 0x0 ref 1 00000100:00100000:0.0:1789760741.641759:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720074003422 time 720073743006 state 0x1 ref 1 00000100:00100000:0.0:1789760741.641795:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720074003422 time 720107084394 state 0x0 ref 1 00000100:00100000:0.0:1789760741.641808:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515690752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760741.641952:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22151168, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:0.0:1789760741.641963:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.642020:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.642027:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.642035:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.642054:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.673561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972725 is committed 00002000:00010000:0.0:1789760741.673737:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22151168 -> 22155264 00010000:00010000:0.0:1789760741.673760:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.673775:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.673786:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.673794:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.673802:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.673813:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.673839:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.673847:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e64a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.674160:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515690752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32344us (32675us total) trans 4294972725 rc 0/0 00000100:00100000:1.0:1789760741.674173:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5437 00000100:00100000:1.0:1789760741.674186:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720074003422 time 720107084394 state 0x2 ref 0 00000100:00100000:0.0:1789760741.678971:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760741.679093:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515690880 00000100:00100000:2.0:1789760741.679107:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515690880 00000100:00100000:2.0:1789760741.679180:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720074003422 time 720107084394 state 0x0 ref 1 00000100:00100000:2.0:1789760741.679200:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720107344810 time 720107084394 state 0x1 ref 1 00000100:00100000:2.0:1789760741.679265:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720107344810 time 720144553553 state 0x0 ref 1 00000100:00100000:2.0:1789760741.679280:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515690880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760741.679345:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22155264, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:2.0:1789760741.679356:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.679376:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.679381:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.679389:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.679404:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.693894:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760741.717523:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22155264 -> 22159360 00010000:00010000:0.0:1789760741.717548:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.717563:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.717574:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.717584:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.717590:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.717605:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.717633:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.717639:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e651 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.717800:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515690880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38515us (38893us total) trans 4294972726 rc 0/0 00000100:00100000:0.0:1789760741.717813:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5438 00000100:00100000:0.0:1789760741.717825:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720107344810 time 720144553553 state 0x2 ref 0 00000001:00080000:1.0:1789760741.718133:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972726 is committed 00000100:00100000:0.0:1789760741.721437:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760741.721548:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691008 00000100:00100000:1.0:1789760741.721562:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691008 00000100:00100000:1.0:1789760741.721698:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720107344810 time 720144553553 state 0x0 ref 1 00000100:00100000:1.0:1789760741.722174:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720144813969 time 720144553553 state 0x1 ref 1 00000100:00100000:1.0:1789760741.722246:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720144813969 time 720187534933 state 0x0 ref 1 00000100:00100000:1.0:1789760741.722265:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760741.722337:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22159360, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:1.0:1789760741.722348:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.722368:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760741.722375:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.722383:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.722400:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.777409:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972727 is committed 00002000:00010000:0.0:1789760741.777799:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22159360 -> 22163456 00010000:00010000:0.0:1789760741.777819:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.777834:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.777845:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.777858:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.777863:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.777875:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.777912:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.777922:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e658 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.778053:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55787us (56624us total) trans 4294972727 rc 0/0 00000100:00100000:0.0:1789760741.778065:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5439 00000100:00100000:0.0:1789760741.778078:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720144813969 time 720187534933 state 0x2 ref 0 00000100:00100000:1.0:1789760741.797033:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.798098:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691136 00000100:00100000:0.0:1789760741.798113:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691136 00000100:00100000:0.0:1789760741.798190:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720144813969 time 720187534933 state 0x0 ref 1 00000100:00100000:0.0:1789760741.798209:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720187795349 time 720187534933 state 0x1 ref 1 00000100:00100000:0.0:1789760741.798245:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720187795349 time 720263533905 state 0x0 ref 1 00000100:00100000:0.0:1789760741.798264:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760741.798327:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22163456, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:0.0:1789760741.798338:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.798358:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.798363:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.798371:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.798388:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.831779:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972728 is committed 00002000:00010000:0.0:1789760741.832160:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22163456 -> 22167552 00010000:00010000:0.0:1789760741.832182:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.832198:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.832209:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.832218:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.832224:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.832235:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.832260:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.832269:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e65f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.832443:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34174us (35423us total) trans 4294972728 rc 0/0 00000100:00100000:0.0:1789760741.832456:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5440 00000100:00100000:0.0:1789760741.832541:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720187795349 time 720263533905 state 0x2 ref 0 00000100:00100000:2.0:1789760741.837184:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760741.837275:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691264 00000100:00100000:0.0:1789760741.837289:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691264 00000100:00100000:0.0:1789760741.837361:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720187795349 time 720263533905 state 0x0 ref 1 00000100:00100000:0.0:1789760741.837379:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720263794321 time 720263533905 state 0x1 ref 1 00000100:00100000:0.0:1789760741.837418:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720263794321 time 720302707076 state 0x0 ref 1 00000100:00100000:0.0:1789760741.837431:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515691264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760741.837502:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22167552, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb007 00010000:00010000:0.0:1789760741.837512:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.837533:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.837538:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.837545:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.837562:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.886543:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544411904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.886634:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.886726:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93bfd9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544412416:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.886754:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.886779:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760741.886785:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544412288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.886810:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.886837:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c61f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544412160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.886859:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.886917:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412416 00000100:00100000:2.0:1789760741.886926:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412416 00000100:00100000:2.0:1789760741.886999:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412160 00000100:00100000:2.0:1789760741.887003:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412160 00000100:00100000:2.0:1789760741.887027:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:2.0:1789760741.887036:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544412416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.887105:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544412416:12345-0@lo:400:kworker.0 Request processed in 70us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.887113:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:2.0:1789760741.887130:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789760741.887136:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544412160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.887165:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544412160:12345-0@lo:400:kworker.0 Request processed in 29us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.887171:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00080000:2.0:1789760741.887188:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760741.888301:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c61f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544412160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.888320:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789760741.888327:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c62a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544412032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.888364:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.888392:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93bfd8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544412544:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.888412:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412032 00000100:00100000:3.0:1789760741.888415:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.888417:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412032 00000100:00100000:3.0:1789760741.888446:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412288 00000100:00100000:3.0:1789760741.888450:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412288 00000100:00100000:2.0:1789760741.888454:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760741.888461:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544412032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.888480:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412544 00000100:00100000:3.0:1789760741.888483:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412544 00000100:00100000:2.0:1789760741.888502:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544412032:12345-0@lo:400:kworker.0 Request processed in 42us (138us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760741.888503:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789760741.888509:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1789760741.888509:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544412288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.888527:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544411904 00000100:00100000:2.0:1789760741.888530:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544411904 00000100:00100000:3.0:1789760741.888539:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544412288:12345-0@lo:400:kworker.0 Request processed in 31us (1729us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760741.888545:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789760741.888560:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:2.0:1789760741.888566:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909351850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544411904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.888597:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909351850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544411904:12345-0@lo:400:kworker.0 Request processed in 31us (1964us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.888603:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789760741.888755:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:3.0:1789760741.888763:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544412544:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.888795:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544412544:12345-0@lo:400:kworker.0 Request processed in 33us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760741.888802:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00080000:3.0:1789760741.888820:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760741.888891:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760741.888990:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c62a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544412032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.889002:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93bfd8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544412544:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.889321:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544412288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.889331:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c60700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544411904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.889341:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93bfd9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544412416:0@lo:400:kworker.0 00002000:00010000:0.0:1789760741.895363:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22167552 -> 22171648 00002000:00010000:0.0:1789760741.895372:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760741 -> 1789760742 00002000:00010000:0.0:1789760741.895375:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760741 -> 1789760742 00010000:00010000:0.0:1789760741.895388:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.895399:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.895408:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.895415:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.895420:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.895428:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.895445:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.895453:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e666 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.895542:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515691264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58109us (58370us total) trans 4294972729 rc 0/0 00000100:00100000:0.0:1789760741.895621:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5441 00000100:00100000:0.0:1789760741.895631:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720263794321 time 720302707076 state 0x2 ref 0 00000001:00080000:0.0:1789760741.895803:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972729 is committed 00000100:00100000:1.0:1789760741.901907:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760741.902758:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691392 00000100:00100000:2.0:1789760741.902772:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691392 00000100:00100000:2.0:1789760741.902829:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720263794321 time 720302707076 state 0x0 ref 1 00000100:00100000:2.0:1789760741.902845:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720302967492 time 720302707076 state 0x1 ref 1 00000100:00100000:2.0:1789760741.902884:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720302967492 time 720368173259 state 0x0 ref 1 00000100:00100000:2.0:1789760741.902899:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760741.902949:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22171648, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:2.0:1789760741.902957:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.902972:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.902976:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.902994:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.903008:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.985120:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972730 is committed 00002000:00010000:0.0:1789760741.985231:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22171648 -> 22175744 00010000:00010000:0.0:1789760741.985251:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.985267:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.985277:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.985285:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760741.985292:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.985307:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.985332:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.985341:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e66d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.985568:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 82665us (83681us total) trans 4294972730 rc 0/0 00000100:00100000:0.0:1789760741.985581:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5442 00000100:00100000:0.0:1789760741.985593:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720302967492 time 720368173259 state 0x2 ref 0 00000100:00100000:2.0:1789760741.998163:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.000559:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691520 00000100:00100000:1.0:1789760742.000580:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691520 00000100:00100000:0.0:1789760742.000826:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720302967492 time 720368173259 state 0x0 ref 1 00000100:00100000:0.0:1789760742.000846:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720368433675 time 720368173259 state 0x1 ref 1 00000100:00100000:0.0:1789760742.000875:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720368433675 time 720466164114 state 0x0 ref 1 00000100:00100000:0.0:1789760742.000891:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515691520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.000958:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22175744, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.000970:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.001009:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760742.001015:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.001023:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.001039:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760742.001624:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760742.033256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972731 is committed 00002000:00010000:1.0:1789760742.033374:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22175744 -> 22179840 00010000:00010000:1.0:1789760742.033394:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033409:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033420:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033429:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.033434:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033445:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033470:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.033478:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e674 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.033604:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515691520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32710us (35449us total) trans 4294972731 rc 0/0 00000100:00100000:1.0:1789760742.033617:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5443 00000100:00100000:1.0:1789760742.033629:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720368433675 time 720466164114 state 0x2 ref 0 00000100:00100000:3.0:1789760742.040588:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760742.043152:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691648 00000100:00100000:2.0:1789760742.043181:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691648 00000100:00100000:2.0:1789760742.043266:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720368433675 time 720466164114 state 0x0 ref 1 00000100:00100000:2.0:1789760742.043287:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720466424530 time 720466164114 state 0x1 ref 1 00000100:00100000:0.0:1789760742.043367:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720466424530 time 720508650520 state 0x0 ref 1 00000100:00100000:0.0:1789760742.043388:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.043456:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22179840, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.043693:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.043742:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760742.043747:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.043755:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.043773:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.093884:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972732 is committed 00002000:00010000:0.0:1789760742.094089:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22179840 -> 22183936 00010000:00010000:0.0:1789760742.094110:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.094126:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.094135:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.094143:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760742.094149:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.094159:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.094186:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.094195:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e67b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.094319:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50930us (53739us total) trans 4294972732 rc 0/0 00000100:00100000:0.0:1789760742.094331:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5444 00000100:00100000:0.0:1789760742.094343:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720466424530 time 720508650520 state 0x2 ref 0 00000100:00100000:0.0:1789760742.102164:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.102284:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691776 00000100:00100000:1.0:1789760742.102298:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691776 00000100:00100000:1.0:1789760742.102373:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720466424530 time 720508650520 state 0x0 ref 1 00000100:00100000:1.0:1789760742.102393:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720508910936 time 720508650520 state 0x1 ref 1 00000100:00100000:1.0:1789760742.102466:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720508910936 time 720567754094 state 0x0 ref 1 00000100:00100000:1.0:1789760742.102491:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515691776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.102558:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22183936, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.102568:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.102588:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760742.102593:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.102601:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.102618:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.142530:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22183936 -> 22188032 00010000:00010000:1.0:1789760742.142557:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.142572:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.142582:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.142590:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.142596:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.142606:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.142628:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.142636:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e682 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.142851:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515691776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40367us (40694us total) trans 4294972733 rc 0/0 00000100:00100000:1.0:1789760742.142868:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5445 00000100:00100000:1.0:1789760742.142880:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720508910936 time 720567754094 state 0x2 ref 0 00000001:00080000:3.0:1789760742.144135:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972733 is committed 00000100:00100000:2.0:1789760742.146141:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.146236:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515691904 00000100:00100000:1.0:1789760742.146251:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515691904 00000100:00100000:1.0:1789760742.146325:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720508910936 time 720567754094 state 0x0 ref 1 00000100:00100000:1.0:1789760742.146346:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720568014510 time 720567754094 state 0x1 ref 1 00000100:00100000:1.0:1789760742.146461:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720568014510 time 720611748990 state 0x0 ref 1 00000100:00100000:1.0:1789760742.146477:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515691904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.146542:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22188032, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.146553:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.146573:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760742.146578:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.146585:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.146601:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.185518:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972734 is committed 00002000:00010000:0.0:1789760742.185863:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22188032 -> 22192128 00010000:00010000:0.0:1789760742.185884:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.185899:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.185909:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.185918:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760742.185924:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.185935:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.185957:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.185966:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e689 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.186165:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515691904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39683us (40030us total) trans 4294972734 rc 0/0 00000100:00100000:0.0:1789760742.186177:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5446 00000100:00100000:0.0:1789760742.186189:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720568014510 time 720611748990 state 0x2 ref 0 00000100:00100000:3.0:1789760742.191774:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.191822:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692032 00000100:00100000:0.0:1789760742.191834:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692032 00000100:00100000:0.0:1789760742.191910:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720568014510 time 720611748990 state 0x0 ref 1 00000100:00100000:0.0:1789760742.191928:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720612009406 time 720611748990 state 0x1 ref 1 00000100:00100000:0.0:1789760742.191962:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720612009406 time 720657250766 state 0x0 ref 1 00000100:00100000:0.0:1789760742.191975:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515692032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.192067:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22192128, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.192078:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.192097:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760742.192103:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.192110:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.192125:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.247135:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22192128 -> 22196224 00010000:00010000:1.0:1789760742.247164:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.247180:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.247190:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.247197:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.247204:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.247302:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.247326:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.247334:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e690 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.247855:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515692032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55552us (55766us total) trans 4294972735 rc 0/0 00000100:00100000:1.0:1789760742.247873:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5447 00000100:00100000:1.0:1789760742.247884:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720612009406 time 720657250766 state 0x2 ref 0 00000001:00080000:2.0:1789760742.248869:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972735 is committed 00000100:00100000:1.0:1789760742.252461:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.252566:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692160 00000100:00100000:0.0:1789760742.252576:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692160 00000100:00100000:0.0:1789760742.252631:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720612009406 time 720657250766 state 0x0 ref 1 00000100:00100000:0.0:1789760742.252646:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720657511182 time 720657250766 state 0x1 ref 1 00000100:00100000:0.0:1789760742.252682:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720657511182 time 720717971025 state 0x0 ref 1 00000100:00100000:0.0:1789760742.252694:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515692160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.252776:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22196224, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.252785:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.252800:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760742.252804:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.252810:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.252823:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.301779:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22196224 -> 22200320 00010000:00010000:1.0:1789760742.301798:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301847:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301858:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301865:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.301870:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301878:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301911:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.301920:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e697 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.302025:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515692160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49328us (49568us total) trans 4294972736 rc 0/0 00000100:00100000:1.0:1789760742.302037:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5448 00000100:00100000:1.0:1789760742.302046:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720657511182 time 720717971025 state 0x2 ref 0 00000001:00080000:2.0:1789760742.303276:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972736 is committed 00000100:00100000:3.0:1789760742.305442:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.307616:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692288 00000100:00100000:0.0:1789760742.307630:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692288 00000100:00100000:0.0:1789760742.307695:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720657511182 time 720717971025 state 0x0 ref 1 00000100:00100000:0.0:1789760742.307822:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720718231441 time 720717971025 state 0x1 ref 1 00000100:00100000:0.0:1789760742.308134:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720718231441 time 720773421414 state 0x0 ref 1 00000100:00100000:0.0:1789760742.308151:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515692288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.308199:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22200320, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.308209:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.308225:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760742.308229:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.308236:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.308251:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.320820:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760742.321253:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692416 00000100:00100000:2.0:1789760742.321264:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692416 00000100:00100000:2.0:1789760742.321343:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 392 00000100:00100000:2.0:1789760742.321354:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909351050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515692416:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760742.321423:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909351050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515692416:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 72us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760742.321430:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 392 00000100:00100000:2.0:1789760742.322286:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.322553:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692544 00000100:00100000:1.0:1789760742.322561:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692544 00000100:00100000:1.0:1789760742.323095:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 456 00000100:00100000:1.0:1789760742.323109:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515692544:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760742.323167:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515692544:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 64us (884us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760742.323175:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 456 00000100:00080000:1.0:1789760742.323200:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760742.326881:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.326923:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692672 00000100:00100000:1.0:1789760742.326932:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692672 00000100:00100000:1.0:1789760742.327020:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 457 00000100:00100000:1.0:1789760742.327034:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515692672:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760742.327122:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515692672:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 90us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760742.327131:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 457 00000100:00080000:1.0:1789760742.327306:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:0.0:1789760742.346009:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22200320 -> 22204416 00010000:00010000:0.0:1789760742.346038:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.346054:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.346064:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.346071:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760742.346077:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.346087:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.346117:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.346124:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e69e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.346249:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515692288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38095us (40807us total) trans 4294972737 rc 0/0 00000100:00100000:0.0:1789760742.346261:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5449 00000100:00100000:0.0:1789760742.346272:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720718231441 time 720773421414 state 0x2 ref 0 00000001:00080000:2.0:1789760742.346848:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972737 is committed 00000100:00100000:1.0:1789760742.350238:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760742.350417:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692800 00000100:00100000:2.0:1789760742.350436:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692800 00000100:00100000:2.0:1789760742.350510:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720718231441 time 720773421414 state 0x0 ref 1 00000100:00100000:2.0:1789760742.350529:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720773681830 time 720773421414 state 0x1 ref 1 00000100:00100000:2.0:1789760742.350601:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720773681830 time 720815888902 state 0x0 ref 1 00000100:00100000:2.0:1789760742.350616:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515692800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760742.350684:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22204416, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:2.0:1789760742.350694:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.350762:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760742.350767:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.350775:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.350792:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.394237:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972738 is committed 00002000:00010000:0.0:1789760742.394643:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22204416 -> 22208512 00010000:00010000:0.0:1789760742.394662:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.394684:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.394695:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.394703:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760742.394752:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.394769:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.394795:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.394805:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e6a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.395065:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515692800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44444us (44834us total) trans 4294972738 rc 0/0 00000100:00100000:0.0:1789760742.395079:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5450 00000100:00100000:0.0:1789760742.395092:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720773681830 time 720815888902 state 0x2 ref 0 00000100:00100000:1.0:1789760742.398478:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760742.398653:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515692928 00000100:00100000:2.0:1789760742.398665:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515692928 00000100:00100000:2.0:1789760742.398823:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720773681830 time 720815888902 state 0x0 ref 1 00000100:00100000:2.0:1789760742.398841:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720816149318 time 720815888902 state 0x1 ref 1 00000100:00100000:2.0:1789760742.398858:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720816149318 time 720864148085 state 0x0 ref 1 00000100:00100000:2.0:1789760742.398872:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515692928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760742.399176:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22208512, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:2.0:1789760742.399188:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.399208:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760742.399214:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.399222:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.399238:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.448202:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22208512 -> 22212608 00010000:00010000:1.0:1789760742.448234:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.448252:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.448263:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.448272:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.448278:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.448288:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.448314:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.448323:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e6ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.449066:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972739 is committed 00000100:00100000:1.0:1789760742.449303:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515692928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49577us (50043us total) trans 4294972739 rc 0/0 00000100:00100000:1.0:1789760742.449322:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5451 00000100:00100000:1.0:1789760742.449335:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720816149318 time 720864148085 state 0x2 ref 0 00000100:00100000:1.0:1789760742.454857:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.456428:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693056 00000100:00100000:0.0:1789760742.456451:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693056 00000100:00100000:0.0:1789760742.456538:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720816149318 time 720864148085 state 0x0 ref 1 00000100:00100000:0.0:1789760742.456557:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720864408501 time 720864148085 state 0x1 ref 1 00000100:00100000:0.0:1789760742.456667:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720864408501 time 720921955902 state 0x0 ref 1 00000100:00100000:0.0:1789760742.456683:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515693056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.456775:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22212608, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.456787:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.456808:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760742.456813:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.456822:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.456840:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.521362:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22212608 -> 22216704 00010000:00010000:1.0:1789760742.521389:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.521411:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.521420:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.521428:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.521432:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.521442:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.521465:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.521473:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e6b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.521592:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515693056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64905us (66743us total) trans 4294972740 rc 0/0 00000100:00100000:1.0:1789760742.521605:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5452 00000100:00100000:1.0:1789760742.521618:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720864408501 time 720921955902 state 0x2 ref 0 00000001:00080000:3.0:1789760742.522382:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972740 is committed 00000100:00100000:0.0:1789760742.526474:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.526564:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693184 00000100:00100000:1.0:1789760742.526579:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693184 00000100:00100000:1.0:1789760742.526653:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720864408501 time 720921955902 state 0x0 ref 1 00000100:00100000:1.0:1789760742.526674:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720922216318 time 720921955902 state 0x1 ref 1 00000100:00100000:1.0:1789760742.526817:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720922216318 time 720991997210 state 0x0 ref 1 00000100:00100000:1.0:1789760742.526833:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515693184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.526898:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22216704, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.526908:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.526928:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760742.526932:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.526940:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.526956:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.565662:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22216704 -> 22220800 00010000:00010000:1.0:1789760742.565693:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.565728:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.565745:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.565754:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.565760:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.565770:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.565799:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.565806:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.566024:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515693184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39178us (39556us total) trans 4294972741 rc 0/0 00000100:00100000:1.0:1789760742.566038:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5453 00000100:00100000:1.0:1789760742.566051:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720922216318 time 720991997210 state 0x2 ref 0 00000001:00080000:2.0:1789760742.566885:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972741 is committed 00000100:00100000:2.0:1789760742.574114:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.576690:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693312 00000100:00100000:1.0:1789760742.576727:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693312 00000100:00100000:2.0:1789760742.580103:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720922216318 time 720991997210 state 0x0 ref 1 00000100:00100000:2.0:1789760742.580128:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720992257626 time 720991997210 state 0x1 ref 1 00000100:00100000:2.0:1789760742.580186:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720992257626 time 721045474901 state 0x0 ref 1 00000100:00100000:2.0:1789760742.580216:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760742.580289:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22220800, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:2.0:1789760742.580299:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.580321:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.580326:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.580333:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.580350:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.630867:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22220800 -> 22224896 00010000:00010000:2.0:1789760742.630907:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.630924:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.630935:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.630944:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760742.630949:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.630960:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.630995:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.631004:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e6c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.631120:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515693312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50913us (57013us total) trans 4294972742 rc 0/0 00000100:00100000:2.0:1789760742.631137:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5454 00000100:00100000:2.0:1789760742.631150:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720992257626 time 721045474901 state 0x2 ref 0 00000001:00080000:2.0:1789760742.631197:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972742 is committed 00000100:00100000:3.0:1789760742.641969:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.643109:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693440 00000100:00100000:1.0:1789760742.643126:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693440 00000100:00100000:1.0:1789760742.643203:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 720992257626 time 721045474901 state 0x0 ref 1 00000100:00100000:1.0:1789760742.643223:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721045735317 time 721045474901 state 0x1 ref 1 00000100:00100000:1.0:1789760742.643258:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721045735317 time 721108547240 state 0x0 ref 1 00000100:00100000:1.0:1789760742.643273:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.643338:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22224896, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.643361:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.643381:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.643386:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.643394:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.643411:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.681476:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972743 is committed 00002000:00010000:2.0:1789760742.681578:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22224896 -> 22228992 00010000:00010000:2.0:1789760742.681597:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.681612:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.681621:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.681629:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760742.681636:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.681646:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.681667:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.681675:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e6c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.681831:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38554us (40022us total) trans 4294972743 rc 0/0 00000100:00100000:2.0:1789760742.681841:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5455 00000100:00100000:2.0:1789760742.681853:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721045735317 time 721108547240 state 0x2 ref 0 00000100:00100000:2.0:1789760742.692188:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.692569:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693568 00000100:00100000:0.0:1789760742.692589:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693568 00000100:00100000:0.0:1789760742.692676:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721045735317 time 721108547240 state 0x0 ref 1 00000100:00100000:0.0:1789760742.693255:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721108807656 time 721108547240 state 0x1 ref 1 00000100:00100000:1.0:1789760742.693532:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721108807656 time 721158815646 state 0x0 ref 1 00000100:00100000:1.0:1789760742.693562:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515693568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.693639:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22228992, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.693649:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.693671:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.693678:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.693918:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.694013:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.718112:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760742.743042:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972744 is committed 00002000:00010000:3.0:1789760742.743182:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22228992 -> 22233088 00010000:00010000:3.0:1789760742.743203:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.743219:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.743230:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.743238:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760742.743245:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.743255:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.743278:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.743286:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e6cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.743418:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515693568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49862us (51235us total) trans 4294972744 rc 0/0 00000100:00100000:3.0:1789760742.743430:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5456 00000100:00100000:3.0:1789760742.743443:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721108807656 time 721158815646 state 0x2 ref 0 00000100:00100000:3.0:1789760742.750260:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760742.750541:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693696 00000100:00100000:1.0:1789760742.750553:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693696 00000100:00100000:1.0:1789760742.750617:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721108807656 time 721158815646 state 0x0 ref 1 00000100:00100000:1.0:1789760742.750635:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721159076062 time 721158815646 state 0x1 ref 1 00000100:00100000:1.0:1789760742.750679:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721159076062 time 721215967069 state 0x0 ref 1 00000100:00100000:1.0:1789760742.750692:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515693696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.750794:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22233088, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.750804:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.750821:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.750825:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.750831:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.750844:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.783757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972745 is committed 00002000:00010000:3.0:1789760742.784133:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22233088 -> 22237184 00010000:00010000:3.0:1789760742.784153:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.784168:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.784177:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.784185:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760742.784191:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.784201:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.784225:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.784240:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1e6d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.784465:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515693696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33769us (34221us total) trans 4294972745 rc 0/0 00000100:00100000:3.0:1789760742.785190:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5457 00000100:00100000:3.0:1789760742.785203:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721159076062 time 721215967069 state 0x2 ref 0 00000100:00100000:2.0:1789760742.787792:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.788182:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693824 00000100:00100000:0.0:1789760742.788199:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693824 00000100:00100000:0.0:1789760742.788344:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721159076062 time 721215967069 state 0x0 ref 1 00000100:00100000:0.0:1789760742.789074:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721216227485 time 721215967069 state 0x1 ref 1 00000100:00100000:1.0:1789760742.789193:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721216227485 time 721254478042 state 0x0 ref 1 00000100:00100000:1.0:1789760742.789217:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760742.789302:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22237184, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:1.0:1789760742.789314:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.789334:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760742.789338:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.789345:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.789470:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760742.832106:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22237184 -> 22241280 00000001:00080000:3.0:1789760742.832109:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972746 is committed 00010000:00010000:1.0:1789760742.832136:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.832152:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.832163:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.832173:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760742.832179:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.832189:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.832216:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.832223:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e6dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.832346:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43131us (44562us total) trans 4294972746 rc 0/0 00000100:00100000:1.0:1789760742.832359:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5458 00000100:00100000:1.0:1789760742.833157:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721216227485 time 721254478042 state 0x2 ref 0 00000100:00100000:2.0:1789760742.834901:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.835007:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515693952 00000100:00100000:0.0:1789760742.835022:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515693952 00000100:00100000:0.0:1789760742.835091:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721216227485 time 721254478042 state 0x0 ref 1 00000100:00100000:0.0:1789760742.835105:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721254738458 time 721254478042 state 0x1 ref 1 00000100:00100000:0.0:1789760742.835130:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721254738458 time 721300418175 state 0x0 ref 1 00000100:00100000:0.0:1789760742.835144:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.835203:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22241280, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb007 00010000:00010000:0.0:1789760742.835211:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.835228:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.835233:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.835239:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.835253:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.893885:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972747 is committed 00002000:00010000:2.0:1789760742.893902:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22241280 -> 22245376 00002000:00010000:2.0:1789760742.893910:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760742 -> 1789760743 00002000:00010000:2.0:1789760742.893913:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760742 -> 1789760743 00010000:00010000:2.0:1789760742.893927:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.894024:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.894034:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.894042:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760742.894047:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.894055:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.894076:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.894133:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e6e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.894637:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515693952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59491us (59741us total) trans 4294972747 rc 0/0 00000100:00100000:2.0:1789760742.894648:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5459 00000100:00100000:2.0:1789760742.894657:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721254738458 time 721300418175 state 0x2 ref 0 00000100:00100000:2.0:1789760742.899478:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760742.899624:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694080 00000100:00100000:0.0:1789760742.899634:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694080 00000100:00100000:0.0:1789760742.899688:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721254738458 time 721300418175 state 0x0 ref 1 00000100:00100000:0.0:1789760742.899704:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721300678591 time 721300418175 state 0x1 ref 1 00000100:00100000:0.0:1789760742.899754:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721300678591 time 721365043641 state 0x0 ref 1 00000100:00100000:0.0:1789760742.899767:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760742.899817:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22245376, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:0.0:1789760742.899825:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.899841:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.899845:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.899851:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.899864:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.928421:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972748 is committed 00002000:00010000:2.0:1789760742.928512:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22245376 -> 22249472 00010000:00010000:2.0:1789760742.928535:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.928551:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.928561:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.928569:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760742.928575:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.928591:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.928616:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.928625:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e6eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.928879:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29106us (29402us total) trans 4294972748 rc 0/0 00000100:00100000:2.0:1789760742.929963:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5460 00000100:00100000:2.0:1789760742.930025:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721300678591 time 721365043641 state 0x2 ref 0 00000100:00100000:1.0:1789760742.933161:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760742.933371:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694208 00000100:00100000:2.0:1789760742.933384:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694208 00000100:00100000:2.0:1789760742.933459:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721300678591 time 721365043641 state 0x0 ref 1 00000100:00100000:2.0:1789760742.933478:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721365304057 time 721365043641 state 0x1 ref 1 00000100:00100000:2.0:1789760742.933512:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721365304057 time 721398800817 state 0x0 ref 1 00000100:00100000:2.0:1789760742.933545:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760742.933643:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22249472, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:2.0:1789760742.933656:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.933681:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.933687:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.933695:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.933764:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.966016:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22249472 -> 22253568 00010000:00010000:2.0:1789760742.966045:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.966063:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.966075:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.966083:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760742.966089:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.966099:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.966125:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.966135:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e6f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.966265:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515694208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32734us (33111us total) trans 4294972749 rc 0/0 00000100:00100000:2.0:1789760742.966278:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5461 00000100:00100000:2.0:1789760742.966290:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721365304057 time 721398800817 state 0x2 ref 0 00000001:00080000:3.0:1789760742.966916:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972749 is committed 00000100:00100000:0.0:1789760742.971762:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760742.971984:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694336 00000100:00100000:3.0:1789760742.972062:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694336 00000100:00100000:3.0:1789760742.972138:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721365304057 time 721398800817 state 0x0 ref 1 00000100:00100000:3.0:1789760742.972158:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721399061233 time 721398800817 state 0x1 ref 1 00000100:00100000:3.0:1789760742.972195:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721399061233 time 721437483668 state 0x0 ref 1 00000100:00100000:3.0:1789760742.972210:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515694336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760742.972277:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22253568, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760742.972288:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.972309:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.972315:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.972323:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.972340:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.028628:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22253568 -> 22257664 00010000:00010000:3.0:1789760743.028660:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.028676:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.028687:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.028695:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760743.028701:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.029046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972750 is committed 00010000:00010000:3.0:1789760743.029199:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.029228:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.029243:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e6f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.029368:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515694336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57152us (57662us total) trans 4294972750 rc 0/0 00000100:00100000:3.0:1789760743.029381:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5462 00000100:00100000:3.0:1789760743.029394:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721399061233 time 721437483668 state 0x2 ref 0 00000100:00100000:0.0:1789760743.033218:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.033397:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694464 00000100:00100000:3.0:1789760743.033456:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694464 00000100:00100000:3.0:1789760743.033521:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721399061233 time 721437483668 state 0x0 ref 1 00000100:00100000:3.0:1789760743.033536:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721437744084 time 721437483668 state 0x1 ref 1 00000100:00100000:3.0:1789760743.033566:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721437744084 time 721498854351 state 0x0 ref 1 00000100:00100000:3.0:1789760743.033581:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.033667:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22257664, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760743.033677:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.034092:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760743.034199:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.034209:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.034232:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.077831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972751 is committed 00002000:00010000:0.0:1789760743.078014:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22257664 -> 22261760 00010000:00010000:0.0:1789760743.078035:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.078051:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.078346:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.078356:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760743.078362:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.078373:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.078401:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.078410:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1e700 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.078537:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44950us (45572us total) trans 4294972751 rc 0/0 00000100:00100000:0.0:1789760743.078550:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5463 00000100:00100000:0.0:1789760743.078563:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721437744084 time 721498854351 state 0x2 ref 0 00000100:00100000:2.0:1789760743.083213:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760743.083324:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694592 00000100:00100000:1.0:1789760743.083338:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694592 00000100:00100000:1.0:1789760743.083526:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721437744084 time 721498854351 state 0x0 ref 1 00000100:00100000:1.0:1789760743.083549:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721499114767 time 721498854351 state 0x1 ref 1 00000100:00100000:1.0:1789760743.083625:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721499114767 time 721548913231 state 0x0 ref 1 00000100:00100000:1.0:1789760743.083643:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760743.083766:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22261760, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:1.0:1789760743.083777:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.083799:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760743.083804:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.083812:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.083829:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.135229:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22261760 -> 22265856 00010000:00010000:3.0:1789760743.135257:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.135273:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.135283:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.135292:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760743.135297:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.135307:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.135331:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.135339:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e707 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.135461:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515694592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51816us (52258us total) trans 4294972752 rc 0/0 00000100:00100000:3.0:1789760743.135473:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5464 00000100:00100000:3.0:1789760743.135486:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721499114767 time 721548913231 state 0x2 ref 0 00000001:00080000:0.0:1789760743.137128:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972752 is committed 00000100:00100000:2.0:1789760743.144401:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760743.144700:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694720 00000100:00100000:0.0:1789760743.144767:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694720 00000100:00100000:0.0:1789760743.144843:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721499114767 time 721548913231 state 0x0 ref 1 00000100:00100000:0.0:1789760743.144864:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721549173647 time 721548913231 state 0x1 ref 1 00000100:00100000:0.0:1789760743.144909:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721549173647 time 721610197439 state 0x0 ref 1 00000100:00100000:0.0:1789760743.145022:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515694720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760743.145102:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22265856, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:0.0:1789760743.145115:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.145135:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760743.145140:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.145148:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.145164:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.199469:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972753 is committed 00002000:00010000:2.0:1789760743.201766:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22265856 -> 22269952 00010000:00010000:2.0:1789760743.201794:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.201810:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.201821:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.201829:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.201835:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.201851:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.201877:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.201887:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e70e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.202093:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515694720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57164us (57697us total) trans 4294972753 rc 0/0 00000100:00100000:2.0:1789760743.202107:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5465 00000100:00100000:2.0:1789760743.202119:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721549173647 time 721610197439 state 0x2 ref 0 00000100:00100000:2.0:1789760743.207178:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760743.207341:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694848 00000100:00100000:0.0:1789760743.207355:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694848 00000100:00100000:0.0:1789760743.207437:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721549173647 time 721610197439 state 0x0 ref 1 00000100:00100000:0.0:1789760743.207457:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721610457855 time 721610197439 state 0x1 ref 1 00000100:00100000:0.0:1789760743.207501:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721610457855 time 721672789942 state 0x0 ref 1 00000100:00100000:0.0:1789760743.207516:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760743.207579:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22269952, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:0.0:1789760743.207588:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.207605:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760743.207609:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.207614:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.207628:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.252109:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972754 is committed 00002000:00010000:0.0:1789760743.252529:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22269952 -> 22274048 00010000:00010000:0.0:1789760743.252550:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.252566:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.252577:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.252586:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760743.252592:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.252602:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.252635:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.252645:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e715 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.252823:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515694848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45302us (45653us total) trans 4294972754 rc 0/0 00000100:00100000:0.0:1789760743.252836:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5466 00000100:00100000:0.0:1789760743.252849:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721610457855 time 721672789942 state 0x2 ref 0 00000100:00100000:0.0:1789760743.266542:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.266814:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515694976 00000100:00100000:2.0:1789760743.266827:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515694976 00000100:00100000:3.0:1789760743.267133:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721610457855 time 721672789942 state 0x0 ref 1 00000100:00100000:3.0:1789760743.267334:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721673050358 time 721672789942 state 0x1 ref 1 00000100:00100000:3.0:1789760743.267497:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721673050358 time 721732719896 state 0x0 ref 1 00000100:00100000:3.0:1789760743.267515:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515694976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.267585:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22274048, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760743.267597:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.267617:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760743.267623:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.267631:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.267835:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.312644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972755 is committed 00002000:00010000:2.0:1789760743.313408:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22274048 -> 22278144 00010000:00010000:2.0:1789760743.313665:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.313682:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.313693:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.313742:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.313749:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.313760:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.313788:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.313797:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1e71c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.313935:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515694976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46417us (47803us total) trans 4294972755 rc 0/0 00000100:00100000:2.0:1789760743.313948:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5467 00000100:00100000:2.0:1789760743.313961:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721673050358 time 721732719896 state 0x2 ref 0 00000100:00100000:0.0:1789760743.317397:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.317548:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695104 00000100:00100000:3.0:1789760743.317562:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695104 00000100:00100000:3.0:1789760743.317693:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721673050358 time 721732719896 state 0x0 ref 1 00000100:00100000:3.0:1789760743.317793:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721732980312 time 721732719896 state 0x1 ref 1 00000100:00100000:3.0:1789760743.317909:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721732980312 time 721783111106 state 0x0 ref 1 00000100:00100000:3.0:1789760743.317928:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515695104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.318095:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22278144, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760743.318107:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.318128:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760743.318134:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.318142:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.318158:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.347907:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972756 is committed 00002000:00010000:3.0:1789760743.348443:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22278144 -> 22282240 00010000:00010000:3.0:1789760743.348464:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.348487:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.348498:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.348506:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760743.348511:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.348523:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.348549:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.348557:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e723 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.348796:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515695104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30865us (31406us total) trans 4294972756 rc 0/0 00000100:00100000:3.0:1789760743.348809:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5468 00000100:00100000:3.0:1789760743.348820:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721732980312 time 721783111106 state 0x2 ref 0 00000100:00100000:1.0:1789760743.357277:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.357431:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695232 00000100:00100000:3.0:1789760743.357445:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695232 00000100:00100000:3.0:1789760743.358112:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721732980312 time 721783111106 state 0x0 ref 1 00000100:00100000:3.0:1789760743.358136:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721783371522 time 721783111106 state 0x1 ref 1 00000100:00100000:3.0:1789760743.358172:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721783371522 time 721823460542 state 0x0 ref 1 00000100:00100000:3.0:1789760743.358189:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515695232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.358262:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22282240, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760743.358273:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.358292:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760743.358297:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.358305:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.358323:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.401368:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972757 is committed 00002000:00010000:3.0:1789760743.401752:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22282240 -> 22286336 00010000:00010000:3.0:1789760743.401773:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.401788:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.401804:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.401812:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760743.401819:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.401829:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.401856:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.401864:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e72a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.402011:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515695232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43819us (44741us total) trans 4294972757 rc 0/0 00000100:00100000:3.0:1789760743.402023:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5469 00000100:00100000:3.0:1789760743.402037:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721783371522 time 721823460542 state 0x2 ref 0 00000100:00100000:0.0:1789760743.407392:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760743.407638:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695360 00000100:00100000:1.0:1789760743.407653:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695360 00000100:00100000:1.0:1789760743.407828:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721783371522 time 721823460542 state 0x0 ref 1 00000100:00100000:1.0:1789760743.407849:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721823720958 time 721823460542 state 0x1 ref 1 00000100:00100000:1.0:1789760743.407943:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721823720958 time 721873230638 state 0x0 ref 1 00000100:00100000:1.0:1789760743.407959:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760743.408041:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22286336, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:1.0:1789760743.408055:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.408076:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760743.408082:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.408090:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.408108:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.442838:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22286336 -> 22290432 00010000:00010000:2.0:1789760743.442867:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.442884:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.442905:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.442913:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.442919:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.442929:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.442955:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.442963:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e731 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.443292:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515695360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35329us (35911us total) trans 4294972758 rc 0/0 00000100:00100000:2.0:1789760743.443306:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5470 00000100:00100000:2.0:1789760743.443318:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721823720958 time 721873230638 state 0x2 ref 0 00000001:00080000:2.0:1789760743.443377:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972758 is committed 00000100:00100000:0.0:1789760743.448851:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.449040:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695488 00000100:00100000:2.0:1789760743.449054:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695488 00000100:00100000:2.0:1789760743.449129:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721823720958 time 721873230638 state 0x0 ref 1 00000100:00100000:2.0:1789760743.449152:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721873491054 time 721873230638 state 0x1 ref 1 00000100:00100000:2.0:1789760743.449192:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721873491054 time 721914480692 state 0x0 ref 1 00000100:00100000:2.0:1789760743.449206:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515695488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760743.449268:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22290432, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:2.0:1789760743.449279:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.449299:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760743.449304:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.449311:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.449328:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.491442:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22290432 -> 22294528 00010000:00010000:0.0:1789760743.491470:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.491585:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.491596:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.491604:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760743.491609:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.491620:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.491648:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.491657:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e738 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.491885:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515695488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42673us (43184us total) trans 4294972759 rc 0/0 00000100:00100000:0.0:1789760743.491898:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5471 00000100:00100000:0.0:1789760743.491910:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721873491054 time 721914480692 state 0x2 ref 0 00000001:00080000:3.0:1789760743.492311:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972759 is committed 00000100:00100000:2.0:1789760743.498771:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760743.498834:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695616 00000100:00100000:0.0:1789760743.498849:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695616 00000100:00100000:0.0:1789760743.498923:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721873491054 time 721914480692 state 0x0 ref 1 00000100:00100000:0.0:1789760743.498942:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721914741108 time 721914480692 state 0x1 ref 1 00000100:00100000:0.0:1789760743.498988:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721914741108 time 721964276991 state 0x0 ref 1 00000100:00100000:0.0:1789760743.499005:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760743.499085:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22294528, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:0.0:1789760743.499096:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499116:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760743.499122:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.499129:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.499146:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.527060:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972760 is committed 00002000:00010000:1.0:1789760743.527198:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22294528 -> 22298624 00010000:00010000:1.0:1789760743.527219:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.527233:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.527243:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.527251:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760743.527257:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.527267:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.527300:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.527309:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e73f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.527424:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28417us (28663us total) trans 4294972760 rc 0/0 00000100:00100000:1.0:1789760743.527434:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5472 00000100:00100000:1.0:1789760743.527443:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721914741108 time 721964276991 state 0x2 ref 0 00000100:00100000:0.0:1789760743.531669:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.532426:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695744 00000100:00100000:2.0:1789760743.532442:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695744 00000100:00100000:2.0:1789760743.532700:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721914741108 time 721964276991 state 0x0 ref 1 00000100:00100000:2.0:1789760743.532750:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721964537407 time 721964276991 state 0x1 ref 1 00000100:00100000:2.0:1789760743.532786:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721964537407 time 721998074477 state 0x0 ref 1 00000100:00100000:2.0:1789760743.532803:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760743.532874:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22298624, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:2.0:1789760743.532886:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.532907:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760743.532913:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.532921:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.532939:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.567260:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972761 is committed 00002000:00010000:2.0:1789760743.567406:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22298624 -> 22302720 00010000:00010000:2.0:1789760743.567428:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.567444:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.567454:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.567463:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.567468:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.567479:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.567508:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.567517:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e746 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.567675:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34870us (36070us total) trans 4294972761 rc 0/0 00000100:00100000:2.0:1789760743.567688:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5473 00000100:00100000:2.0:1789760743.567700:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721964537407 time 721998074477 state 0x2 ref 0 00000100:00100000:0.0:1789760743.571130:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.571232:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515695872 00000100:00100000:3.0:1789760743.571245:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515695872 00000100:00100000:3.0:1789760743.571312:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721964537407 time 721998074477 state 0x0 ref 1 00000100:00100000:3.0:1789760743.571330:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721998334893 time 721998074477 state 0x1 ref 1 00000100:00100000:3.0:1789760743.571395:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721998334893 time 722036683160 state 0x0 ref 1 00000100:00100000:3.0:1789760743.571409:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.571472:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22302720, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760743.571483:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.571502:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.571507:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.571515:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.571532:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.607879:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972762 is committed 00002000:00010000:2.0:1789760743.608074:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22302720 -> 22306816 00010000:00010000:2.0:1789760743.608094:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.608110:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.608120:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.608129:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.608135:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.608145:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.608169:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.608177:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1e74d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.608303:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515695872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36890us (37182us total) trans 4294972762 rc 0/0 00000100:00100000:2.0:1789760743.608317:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5474 00000100:00100000:2.0:1789760743.608330:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721998334893 time 722036683160 state 0x2 ref 0 00000100:00100000:1.0:1789760743.612401:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.612553:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696000 00000100:00100000:3.0:1789760743.612567:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696000 00000100:00100000:3.0:1789760743.612693:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 721998334893 time 722036683160 state 0x0 ref 1 00000100:00100000:3.0:1789760743.613008:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722036943576 time 722036683160 state 0x1 ref 1 00000100:00100000:1.0:1789760743.613106:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722036943576 time 722078391689 state 0x0 ref 1 00000100:00100000:1.0:1789760743.613125:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760743.613191:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22306816, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:1.0:1789760743.613203:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.613224:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.613229:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.613236:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.613253:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.641836:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972763 is committed 00002000:00010000:3.0:1789760743.641964:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22306816 -> 22310912 00010000:00010000:3.0:1789760743.641996:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.642012:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.642021:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.642324:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760743.642331:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.642365:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.642392:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.642401:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e754 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.642518:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29393us (30125us total) trans 4294972763 rc 0/0 00000100:00100000:3.0:1789760743.642529:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5475 00000100:00100000:3.0:1789760743.642540:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722036943576 time 722078391689 state 0x2 ref 0 00000100:00100000:1.0:1789760743.646802:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.647519:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696128 00000100:00100000:2.0:1789760743.647535:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696128 00000100:00100000:2.0:1789760743.647635:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722036943576 time 722078391689 state 0x0 ref 1 00000100:00100000:2.0:1789760743.647657:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722078652105 time 722078391689 state 0x1 ref 1 00000100:00100000:2.0:1789760743.647693:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722078652105 time 722112981747 state 0x0 ref 1 00000100:00100000:2.0:1789760743.647735:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515696128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760743.647804:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22310912, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:2.0:1789760743.647816:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.647836:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.647842:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.647850:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.647867:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.684304:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22310912 -> 22315008 00010000:00010000:3.0:1789760743.684332:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.684348:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.684359:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.684367:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760743.684372:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.684382:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.684406:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.684414:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e75b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.684541:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515696128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36829us (37838us total) trans 4294972764 rc 0/0 00000100:00100000:3.0:1789760743.684554:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5476 00000100:00100000:3.0:1789760743.684567:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722078652105 time 722112981747 state 0x2 ref 0 00000001:00080000:0.0:1789760743.684748:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972764 is committed 00000100:00100000:0.0:1789760743.691593:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.691873:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696256 00000100:00100000:2.0:1789760743.691889:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696256 00000100:00100000:2.0:1789760743.691974:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722078652105 time 722112981747 state 0x0 ref 1 00000100:00100000:2.0:1789760743.691995:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722113242163 time 722112981747 state 0x1 ref 1 00000100:00100000:2.0:1789760743.692034:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722113242163 time 722157322122 state 0x0 ref 1 00000100:00100000:2.0:1789760743.692048:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760743.692117:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22315008, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:2.0:1789760743.692127:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.692150:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.692156:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.692164:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.692181:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.735849:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972765 is committed 00002000:00010000:0.0:1789760743.736100:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22315008 -> 22319104 00010000:00010000:0.0:1789760743.736121:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.736136:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.736146:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.736154:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760743.736159:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.736169:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.736200:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.736209:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e762 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.736348:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44294us (44762us total) trans 4294972765 rc 0/0 00000100:00100000:0.0:1789760743.736362:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5477 00000100:00100000:0.0:1789760743.736375:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722113242163 time 722157322122 state 0x2 ref 0 00000100:00100000:3.0:1789760743.740498:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760743.740596:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696384 00000100:00100000:0.0:1789760743.740666:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696384 00000100:00100000:0.0:1789760743.740810:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722113242163 time 722157322122 state 0x0 ref 1 00000100:00100000:0.0:1789760743.740829:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722157582538 time 722157322122 state 0x1 ref 1 00000100:00100000:0.0:1789760743.740906:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722157582538 time 722206194136 state 0x0 ref 1 00000100:00100000:0.0:1789760743.740921:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515696384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760743.740998:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22319104, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:0.0:1789760743.741011:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.741030:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760743.741035:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.741043:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.741057:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.742143:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760743.776257:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972766 is committed 00002000:00010000:0.0:1789760743.776475:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22319104 -> 22323200 00010000:00010000:0.0:1789760743.776499:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.776513:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.776523:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.776531:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760743.776537:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.776548:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.776575:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.776582:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e769 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.776869:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515696384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35943us (36377us total) trans 4294972766 rc 0/0 00000100:00100000:0.0:1789760743.776881:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5478 00000100:00100000:0.0:1789760743.776902:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722157582538 time 722206194136 state 0x2 ref 0 00000100:00100000:1.0:1789760743.780921:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.781123:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696512 00000100:00100000:3.0:1789760743.781136:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696512 00000100:00100000:3.0:1789760743.781208:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722157582538 time 722206194136 state 0x0 ref 1 00000100:00100000:3.0:1789760743.781227:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722206454552 time 722206194136 state 0x1 ref 1 00000100:00100000:3.0:1789760743.781295:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722206454552 time 722246583761 state 0x0 ref 1 00000100:00100000:3.0:1789760743.781311:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.781372:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22323200, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:3.0:1789760743.781382:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.781400:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.781406:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.781413:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.781430:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.810408:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22323200 -> 22327296 00010000:00010000:2.0:1789760743.810435:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.810451:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.810461:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.810470:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.810476:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.810486:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.810511:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.810526:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1e770 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.810653:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515696512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29338us (29938us total) trans 4294972767 rc 0/0 00000100:00100000:2.0:1789760743.810665:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5479 00000100:00100000:2.0:1789760743.810677:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722206454552 time 722246583761 state 0x2 ref 0 00000001:00080000:3.0:1789760743.810787:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972767 is committed 00000100:00100000:2.0:1789760743.813873:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760743.814101:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696640 00000100:00100000:1.0:1789760743.814114:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696640 00000100:00100000:1.0:1789760743.814187:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722206454552 time 722246583761 state 0x0 ref 1 00000100:00100000:1.0:1789760743.814207:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722246844177 time 722246583761 state 0x1 ref 1 00000100:00100000:1.0:1789760743.814243:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722246844177 time 722279532424 state 0x0 ref 1 00000100:00100000:1.0:1789760743.814258:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515696640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760743.814324:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22327296, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb007 00010000:00010000:1.0:1789760743.814334:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.814354:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.814359:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.814367:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.814385:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.842088:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972768 is committed 00002000:00010000:1.0:1789760743.842313:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22327296 -> 22331392 00002000:00010000:1.0:1789760743.842318:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760743 -> 1789760744 00002000:00010000:1.0:1789760743.842321:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760743 -> 1789760744 00010000:00010000:1.0:1789760743.842336:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.842350:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.842360:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.842368:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760743.842373:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.842384:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.842412:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.842419:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e777 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.842524:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515696640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28264us (28660us total) trans 4294972768 rc 0/0 00000100:00100000:1.0:1789760743.842533:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5480 00000100:00100000:1.0:1789760743.842542:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722246844177 time 722279532424 state 0x2 ref 0 00000100:00100000:0.0:1789760743.847017:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.847138:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696768 00000100:00100000:3.0:1789760743.847153:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696768 00000100:00100000:3.0:1789760743.847230:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722246844177 time 722279532424 state 0x0 ref 1 00000100:00100000:3.0:1789760743.847250:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722279792840 time 722279532424 state 0x1 ref 1 00000100:00100000:3.0:1789760743.847283:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722279792840 time 722312571071 state 0x0 ref 1 00000100:00100000:3.0:1789760743.847300:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.847368:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22331392, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760743.847379:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.847399:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.847404:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.847412:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.847429:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.883870:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972769 is committed 00002000:00010000:2.0:1789760743.884053:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22331392 -> 22335488 00010000:00010000:2.0:1789760743.884078:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.884093:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.884103:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.884112:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.884118:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.884127:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.884152:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.884160:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1e77e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.884282:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515696768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36980us (37271us total) trans 4294972769 rc 0/0 00000100:00100000:2.0:1789760743.884293:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5481 00000100:00100000:2.0:1789760743.884306:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722279792840 time 722312571071 state 0x2 ref 0 00000100:00100000:2.0:1789760743.890930:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760743.891098:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515696896 00000100:00100000:3.0:1789760743.891111:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515696896 00000100:00100000:3.0:1789760743.891187:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722279792840 time 722312571071 state 0x0 ref 1 00000100:00100000:3.0:1789760743.891208:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722312831487 time 722312571071 state 0x1 ref 1 00000100:00100000:3.0:1789760743.891243:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722312831487 time 722356532131 state 0x0 ref 1 00000100:00100000:3.0:1789760743.891257:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515696896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.891323:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22335488, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760743.891335:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.891356:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.891361:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.891369:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.891385:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.925874:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22335488 -> 22339584 00010000:00010000:2.0:1789760743.926130:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926142:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926150:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926155:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760743.926159:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926170:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926190:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.926246:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1e785 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.926385:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515696896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35126us (35467us total) trans 4294972770 rc 0/0 00000100:00100000:2.0:1789760743.926396:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5482 00000100:00100000:2.0:1789760743.926405:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722312831487 time 722356532131 state 0x2 ref 0 00000001:00080000:1.0:1789760743.926474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972770 is committed 00000100:00100000:0.0:1789760743.934154:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760743.934186:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c59500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544412672:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760743.934297:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760743.934406:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544412800:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760743.934600:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760743.935167:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412672 00000100:00100000:2.0:1789760743.935179:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412672 00000100:00100000:3.0:1789760743.935285:0:11703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412800 00000100:00100000:3.0:1789760743.935290:0:11703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412800 00000100:00100000:3.0:1789760743.935408:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789760743.935419:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544412800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760743.935450:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526319616 pending 0 free 3835691008 avail 3824156672 00000100:00100000:3.0:1789760743.935531:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544412800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 113us (930us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.935538:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1789760743.935637:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544412800:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789760743.935765:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760743.935770:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760743.935866:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1789760743.935873:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544412672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760743.935908:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760743.936070:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544412672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 196us (1772us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.936077:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789760743.936151:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c59500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544412672:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760743.936477:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.936549:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697024 00000100:00100000:2.0:1789760743.936555:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697024 00000100:00100000:2.0:1789760743.936604:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722312831487 time 722356532131 state 0x0 ref 1 00000100:00100000:2.0:1789760743.936625:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722356792547 time 722356532131 state 0x1 ref 1 00000100:00100000:2.0:1789760743.936690:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722356792547 time 722401977688 state 0x0 ref 1 00000100:00100000:2.0:1789760743.936702:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515697024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760743.936856:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22339584, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760743.936867:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.936888:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.936894:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.936901:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.936918:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.973093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972771 is committed 00002000:00010000:1.0:1789760743.973243:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22339584 -> 22343680 00010000:00010000:1.0:1789760743.973266:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.973282:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.973293:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.973301:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760743.973307:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.973318:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.973346:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.973354:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1e78c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.973479:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515697024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36771us (36997us total) trans 4294972771 rc 0/0 00000100:00100000:1.0:1789760743.973492:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5483 00000100:00100000:1.0:1789760743.973504:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722356792547 time 722401977688 state 0x2 ref 0 00000100:00100000:0.0:1789760743.978623:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760743.978784:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697152 00000100:00100000:2.0:1789760743.978798:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697152 00000100:00100000:2.0:1789760743.978867:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722356792547 time 722401977688 state 0x0 ref 1 00000100:00100000:2.0:1789760743.978898:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722402238104 time 722401977688 state 0x1 ref 1 00000100:00100000:2.0:1789760743.978932:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722402238104 time 722444220825 state 0x0 ref 1 00000100:00100000:2.0:1789760743.978946:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760743.979029:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22343680, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760743.979041:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.979060:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.979065:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.979072:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.979093:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.029609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972772 is committed 00002000:00010000:1.0:1789760744.029796:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22343680 -> 22347776 00010000:00010000:1.0:1789760744.029816:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.029833:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.029844:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.029852:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760744.029857:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.029868:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.030011:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.030026:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e793 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.030160:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51209us (51544us total) trans 4294972772 rc 0/0 00000100:00100000:1.0:1789760744.030173:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5484 00000100:00100000:1.0:1789760744.030185:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722402238104 time 722444220825 state 0x2 ref 0 00000100:00100000:2.0:1789760744.034385:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.037022:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697280 00000100:00100000:0.0:1789760744.037044:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697280 00000100:00100000:1.0:1789760744.037788:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722402238104 time 722444220825 state 0x0 ref 1 00000100:00100000:1.0:1789760744.037809:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722444481241 time 722444220825 state 0x1 ref 1 00000100:00100000:1.0:1789760744.037850:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722444481241 time 722503138768 state 0x0 ref 1 00000100:00100000:1.0:1789760744.037867:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515697280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760744.037951:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22347776, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:1.0:1789760744.037972:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.037993:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760744.037999:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.038007:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.038024:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760744.039666:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760744.080202:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972773 is committed 00002000:00010000:1.0:1789760744.080346:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22347776 -> 22351872 00010000:00010000:1.0:1789760744.080368:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.080383:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.080393:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.080401:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760744.080407:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.080417:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.080444:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.080452:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e79a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.080690:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515697280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42819us (46317us total) trans 4294972773 rc 0/0 00000100:00100000:1.0:1789760744.080767:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5485 00000100:00100000:1.0:1789760744.080782:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722444481241 time 722503138768 state 0x2 ref 0 00000100:00100000:2.0:1789760744.084534:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760744.087216:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697408 00000100:00100000:3.0:1789760744.087244:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697408 00000100:00100000:3.0:1789760744.087342:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722444481241 time 722503138768 state 0x0 ref 1 00000100:00100000:3.0:1789760744.087367:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722503399184 time 722503138768 state 0x1 ref 1 00000100:00100000:3.0:1789760744.087415:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722503399184 time 722552704309 state 0x0 ref 1 00000100:00100000:3.0:1789760744.087778:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760744.087851:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22351872, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760744.087862:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.087894:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760744.087900:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.087909:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.087925:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.145471:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972774 is committed 00002000:00010000:3.0:1789760744.145644:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22351872 -> 22355968 00010000:00010000:3.0:1789760744.145664:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.145680:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.145690:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.145700:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760744.145738:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.145750:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.145775:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.145784:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.145913:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58137us (61385us total) trans 4294972774 rc 0/0 00000100:00100000:3.0:1789760744.145926:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5486 00000100:00100000:3.0:1789760744.145938:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722503399184 time 722552704309 state 0x2 ref 0 00000100:00100000:1.0:1789760744.153051:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760744.153242:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697536 00000100:00100000:2.0:1789760744.153256:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697536 00000100:00100000:2.0:1789760744.153345:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722503399184 time 722552704309 state 0x0 ref 1 00000100:00100000:2.0:1789760744.153365:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722552964725 time 722552704309 state 0x1 ref 1 00000100:00100000:2.0:1789760744.153400:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722552964725 time 722618688600 state 0x0 ref 1 00000100:00100000:2.0:1789760744.153417:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515697536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.153493:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22355968, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.153504:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.153614:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760744.153621:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.153630:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.153649:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.188282:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972775 is committed 00002000:00010000:1.0:1789760744.188459:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22355968 -> 22360064 00010000:00010000:1.0:1789760744.188480:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.188496:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.188506:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.188513:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760744.188518:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.188528:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.188554:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.188562:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e7a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.188770:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515697536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35347us (35726us total) trans 4294972775 rc 0/0 00000100:00100000:1.0:1789760744.188783:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5487 00000100:00100000:1.0:1789760744.188794:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722552964725 time 722618688600 state 0x2 ref 0 00000100:00100000:2.0:1789760744.192567:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760744.195338:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697664 00000100:00100000:3.0:1789760744.195437:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697664 00000100:00100000:3.0:1789760744.195661:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722552964725 time 722618688600 state 0x0 ref 1 00000100:00100000:3.0:1789760744.195687:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722618949016 time 722618688600 state 0x1 ref 1 00000100:00100000:3.0:1789760744.195746:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722618949016 time 722661034124 state 0x0 ref 1 00000100:00100000:3.0:1789760744.195774:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515697664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760744.195841:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22360064, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760744.195852:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.195873:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760744.195878:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.195886:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.195902:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.234697:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972776 is committed 00002000:00010000:2.0:1789760744.234753:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22360064 -> 22364160 00010000:00010000:2.0:1789760744.234773:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.234797:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.234808:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.234817:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760744.234823:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.234833:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.234857:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.234866:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1e7af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.235000:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515697664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39232us (42438us total) trans 4294972776 rc 0/0 00000100:00100000:2.0:1789760744.235013:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5488 00000100:00100000:2.0:1789760744.235025:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722618949016 time 722661034124 state 0x2 ref 0 00000100:00100000:2.0:1789760744.237827:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.237934:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697792 00000100:00100000:0.0:1789760744.237948:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697792 00000100:00100000:0.0:1789760744.238030:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722618949016 time 722661034124 state 0x0 ref 1 00000100:00100000:0.0:1789760744.238051:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722661294540 time 722661034124 state 0x1 ref 1 00000100:00100000:0.0:1789760744.238087:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722661294540 time 722703375719 state 0x0 ref 1 00000100:00100000:0.0:1789760744.238103:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760744.238172:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22364160, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:0.0:1789760744.238183:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.238204:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760744.238210:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.238219:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.238236:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.270431:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22364160 -> 22368256 00010000:00010000:0.0:1789760744.270460:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.270476:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.270486:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.270494:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.270499:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.270509:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.270536:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.270545:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1e7b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.270674:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515697792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32566us (32960us total) trans 4294972777 rc 0/0 00000100:00100000:0.0:1789760744.270686:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5489 00000100:00100000:0.0:1789760744.270698:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722661294540 time 722703375719 state 0x2 ref 0 00000001:00080000:3.0:1789760744.270841:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972777 is committed 00000100:00100000:3.0:1789760744.273692:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760744.273933:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515697920 00000100:00100000:2.0:1789760744.273947:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515697920 00000100:00100000:2.0:1789760744.274028:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722661294540 time 722703375719 state 0x0 ref 1 00000100:00100000:2.0:1789760744.274047:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722703636135 time 722703375719 state 0x1 ref 1 00000100:00100000:2.0:1789760744.274080:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722703636135 time 722739369008 state 0x0 ref 1 00000100:00100000:2.0:1789760744.274094:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.274164:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22368256, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.274175:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.274195:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760744.274201:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.274208:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.274225:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.305514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972778 is committed 00002000:00010000:3.0:1789760744.305728:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22368256 -> 22372352 00010000:00010000:3.0:1789760744.305750:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.305766:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.305776:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.305784:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760744.305790:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.305800:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.305824:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.305831:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e7bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.305973:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515697920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31868us (32284us total) trans 4294972778 rc 0/0 00000100:00100000:3.0:1789760744.305986:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5490 00000100:00100000:3.0:1789760744.305998:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722703636135 time 722739369008 state 0x2 ref 0 00000100:00100000:2.0:1789760744.309158:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760744.309243:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698048 00000100:00100000:1.0:1789760744.309257:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698048 00000100:00100000:1.0:1789760744.309333:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722703636135 time 722739369008 state 0x0 ref 1 00000100:00100000:1.0:1789760744.309353:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722739629424 time 722739369008 state 0x1 ref 1 00000100:00100000:1.0:1789760744.309388:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722739629424 time 722774676335 state 0x0 ref 1 00000100:00100000:1.0:1789760744.309402:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515698048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760744.309468:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22372352, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:1.0:1789760744.309479:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.309500:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760744.309505:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.309513:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.309530:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.346697:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22372352 -> 22376448 00010000:00010000:0.0:1789760744.346749:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.346765:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.346779:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.346787:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.346793:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.346809:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.346820:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972779 is committed 00010000:00010000:0.0:1789760744.346835:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.346843:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e7c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.347074:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515698048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37667us (37924us total) trans 4294972779 rc 0/0 00000100:00100000:0.0:1789760744.347086:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5491 00000100:00100000:0.0:1789760744.347098:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722739629424 time 722774676335 state 0x2 ref 0 00000100:00100000:0.0:1789760744.353954:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760744.354475:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698176 00000100:00100000:3.0:1789760744.354491:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698176 00000100:00100000:3.0:1789760744.354567:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722739629424 time 722774676335 state 0x0 ref 1 00000100:00100000:3.0:1789760744.354587:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722774936751 time 722774676335 state 0x1 ref 1 00000100:00100000:3.0:1789760744.354626:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722774936751 time 722819913717 state 0x0 ref 1 00000100:00100000:3.0:1789760744.354640:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515698176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760744.354744:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22376448, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760744.354755:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.354776:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760744.354782:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.354790:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.354808:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.384523:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972780 is committed 00002000:00010000:2.0:1789760744.384629:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22376448 -> 22380544 00010000:00010000:2.0:1789760744.384651:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.384673:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.384684:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.384692:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760744.384698:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.384732:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.384759:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.384764:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1e7cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.384868:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515698176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30223us (30924us total) trans 4294972780 rc 0/0 00000100:00100000:2.0:1789760744.384891:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5492 00000100:00100000:2.0:1789760744.384904:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722774936751 time 722819913717 state 0x2 ref 0 00000100:00100000:3.0:1789760744.391195:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760744.391331:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698304 00000100:00100000:1.0:1789760744.391345:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698304 00000100:00100000:1.0:1789760744.391420:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722774936751 time 722819913717 state 0x0 ref 1 00000100:00100000:1.0:1789760744.391438:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722820174133 time 722819913717 state 0x1 ref 1 00000100:00100000:1.0:1789760744.391506:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722820174133 time 722856794673 state 0x0 ref 1 00000100:00100000:1.0:1789760744.391521:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515698304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760744.391586:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22380544, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:1.0:1789760744.391597:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.391617:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760744.391622:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.391630:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.391647:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.434493:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972781 is committed 00002000:00010000:0.0:1789760744.434671:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22380544 -> 22384640 00010000:00010000:0.0:1789760744.434692:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.434734:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.434745:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.434759:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.434765:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.434775:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.434798:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.434807:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e7d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.434948:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515698304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43423us (43760us total) trans 4294972781 rc 0/0 00000100:00100000:0.0:1789760744.434973:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5493 00000100:00100000:0.0:1789760744.434987:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722820174133 time 722856794673 state 0x2 ref 0 00000100:00100000:1.0:1789760744.438315:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760744.438422:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698432 00000100:00100000:2.0:1789760744.438437:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698432 00000100:00100000:2.0:1789760744.438506:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722820174133 time 722856794673 state 0x0 ref 1 00000100:00100000:2.0:1789760744.438611:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722857055089 time 722856794673 state 0x1 ref 1 00000100:00100000:2.0:1789760744.438648:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722857055089 time 722903936925 state 0x0 ref 1 00000100:00100000:2.0:1789760744.438663:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515698432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.438765:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22384640, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.438776:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.438797:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760744.438803:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.438811:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.438828:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.469921:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972782 is committed 00002000:00010000:2.0:1789760744.470155:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22384640 -> 22388736 00010000:00010000:2.0:1789760744.470178:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.470194:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.470211:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.470220:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760744.470226:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.470237:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.470264:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.470272:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.470401:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515698432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31733us (32094us total) trans 4294972782 rc 0/0 00000100:00100000:2.0:1789760744.470414:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5494 00000100:00100000:2.0:1789760744.470424:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722857055089 time 722903936925 state 0x2 ref 0 00000100:00100000:2.0:1789760744.473676:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.473826:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698560 00000100:00100000:0.0:1789760744.473842:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698560 00000100:00100000:0.0:1789760744.473926:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722857055089 time 722903936925 state 0x0 ref 1 00000100:00100000:0.0:1789760744.473945:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722904197341 time 722903936925 state 0x1 ref 1 00000100:00100000:0.0:1789760744.474074:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722904197341 time 722939362019 state 0x0 ref 1 00000100:00100000:0.0:1789760744.474093:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515698560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760744.474157:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22388736, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:0.0:1789760744.474167:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.474188:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760744.474193:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.474201:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.474219:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.498783:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972783 is committed 00002000:00010000:0.0:1789760744.498924:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22388736 -> 22392832 00010000:00010000:0.0:1789760744.498945:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.498969:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.498980:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.498988:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.498994:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.499006:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.499031:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.499038:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.499202:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515698560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25104us (25589us total) trans 4294972783 rc 0/0 00000100:00100000:0.0:1789760744.499214:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5495 00000100:00100000:0.0:1789760744.499226:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722904197341 time 722939362019 state 0x2 ref 0 00000100:00100000:3.0:1789760744.502256:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760744.502368:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698688 00000100:00100000:2.0:1789760744.502381:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698688 00000100:00100000:2.0:1789760744.502453:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722904197341 time 722939362019 state 0x0 ref 1 00000100:00100000:2.0:1789760744.502476:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722939622435 time 722939362019 state 0x1 ref 1 00000100:00100000:2.0:1789760744.502511:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722939622435 time 722967800052 state 0x0 ref 1 00000100:00100000:2.0:1789760744.502525:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515698688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.502591:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22392832, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.502602:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.502632:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760744.502638:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.502645:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.502661:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.539558:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22392832 -> 22396928 00010000:00010000:0.0:1789760744.539586:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.539614:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.539626:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.539639:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.539645:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.539657:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.539694:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.539703:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e7e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.539856:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515698688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37326us (37609us total) trans 4294972784 rc 0/0 00000100:00100000:0.0:1789760744.539868:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5496 00000001:00080000:1.0:1789760744.539889:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972784 is committed 00000100:00100000:0.0:1789760744.540018:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722939622435 time 722967800052 state 0x2 ref 0 00000100:00100000:1.0:1789760744.547201:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.547270:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698816 00000100:00100000:0.0:1789760744.547423:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698816 00000100:00100000:0.0:1789760744.547782:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722939622435 time 722967800052 state 0x0 ref 1 00000100:00100000:0.0:1789760744.547803:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722968060468 time 722967800052 state 0x1 ref 1 00000100:00100000:0.0:1789760744.547900:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722968060468 time 723013187435 state 0x0 ref 1 00000100:00100000:0.0:1789760744.547914:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caa050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515698816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760744.547995:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22396928, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:0.0:1789760744.548006:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.548027:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.548032:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.548040:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.548057:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.605456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972785 is committed 00002000:00010000:0.0:1789760744.605511:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22396928 -> 22401024 00010000:00010000:0.0:1789760744.605532:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.605547:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.605556:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.605562:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.605567:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.605574:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.605598:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.605685:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e7ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.606139:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caa050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515698816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58219us (58944us total) trans 4294972785 rc 0/0 00000100:00100000:0.0:1789760744.606152:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5497 00000100:00100000:0.0:1789760744.606164:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722968060468 time 723013187435 state 0x2 ref 0 00000100:00100000:2.0:1789760744.613570:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760744.614116:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515698944 00000100:00100000:3.0:1789760744.614144:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515698944 00000100:00100000:3.0:1789760744.614230:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 722968060468 time 723013187435 state 0x0 ref 1 00000100:00100000:3.0:1789760744.614250:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723013447851 time 723013187435 state 0x1 ref 1 00000100:00100000:3.0:1789760744.614519:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723013447851 time 723079806196 state 0x0 ref 1 00000100:00100000:3.0:1789760744.614535:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515698944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760744.614602:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22401024, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:3.0:1789760744.614622:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.614643:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.614649:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.614656:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.614672:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.659577:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22401024 -> 22405120 00010000:00010000:0.0:1789760744.659605:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.659621:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.659632:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.659640:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.659646:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.659662:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.659698:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.659724:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e7f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.659838:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515698944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45301us (46272us total) trans 4294972786 rc 0/0 00000100:00100000:0.0:1789760744.659851:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5498 00000100:00100000:0.0:1789760744.659863:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723013447851 time 723079806196 state 0x2 ref 0 00000001:00080000:2.0:1789760744.660463:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972786 is committed 00000100:00100000:3.0:1789760744.663530:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.663668:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699072 00000100:00100000:0.0:1789760744.663682:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699072 00000100:00100000:0.0:1789760744.663807:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723013447851 time 723079806196 state 0x0 ref 1 00000100:00100000:0.0:1789760744.663828:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723080066612 time 723079806196 state 0x1 ref 1 00000100:00100000:0.0:1789760744.663866:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723080066612 time 723129154935 state 0x0 ref 1 00000100:00100000:0.0:1789760744.663889:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515699072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760744.663968:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22405120, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:0.0:1789760744.663981:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.664249:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.664255:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.664264:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.664283:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.713456:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22405120 -> 22409216 00010000:00010000:0.0:1789760744.713486:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.713508:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.713519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972787 is committed 00010000:00010000:0.0:1789760744.713586:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.713595:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.713667:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.713679:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.713789:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.713800:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e7fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.713919:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515699072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50034us (50397us total) trans 4294972787 rc 0/0 00000100:00100000:0.0:1789760744.713932:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5499 00000100:00100000:0.0:1789760744.713945:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723080066612 time 723129154935 state 0x2 ref 0 00000100:00100000:0.0:1789760744.716819:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760744.717277:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699200 00000100:00100000:2.0:1789760744.717292:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699200 00000100:00100000:2.0:1789760744.717367:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723080066612 time 723129154935 state 0x0 ref 1 00000100:00100000:2.0:1789760744.717387:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723129415351 time 723129154935 state 0x1 ref 1 00000100:00100000:2.0:1789760744.717416:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723129415351 time 723182704615 state 0x0 ref 1 00000100:00100000:2.0:1789760744.717430:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515699200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.717495:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22409216, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.717504:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.717624:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.717629:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.717638:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.717656:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.755449:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972788 is committed 00002000:00010000:0.0:1789760744.755491:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22409216 -> 22413312 00010000:00010000:0.0:1789760744.755511:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.755526:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.755535:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.755540:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760744.755545:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.755552:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.755577:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.755583:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e803 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.755836:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515699200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38400us (39024us total) trans 4294972788 rc 0/0 00000100:00100000:0.0:1789760744.755851:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5500 00000100:00100000:0.0:1789760744.755864:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723129415351 time 723182704615 state 0x2 ref 0 00000100:00100000:2.0:1789760744.760436:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.760579:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699328 00000100:00100000:0.0:1789760744.760594:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699328 00000100:00100000:0.0:1789760744.760668:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723129415351 time 723182704615 state 0x0 ref 1 00000100:00100000:0.0:1789760744.760686:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723182965031 time 723182704615 state 0x1 ref 1 00000100:00100000:0.0:1789760744.760769:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723182965031 time 723226057447 state 0x0 ref 1 00000100:00100000:0.0:1789760744.760782:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515699328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760744.760842:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22413312, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:0.0:1789760744.760852:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.760872:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.760878:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.760886:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.760901:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.766921:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760744.796101:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972789 is committed 00002000:00010000:3.0:1789760744.796215:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22413312 -> 22417408 00010000:00010000:3.0:1789760744.796240:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.796255:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.796265:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.796274:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760744.796280:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.796289:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.796319:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.796328:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1e80a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.796696:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515699328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35907us (36268us total) trans 4294972789 rc 0/0 00000100:00100000:3.0:1789760744.796771:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5501 00000100:00100000:3.0:1789760744.796784:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723182965031 time 723226057447 state 0x2 ref 0 00000100:00100000:3.0:1789760744.801570:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760744.801832:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699456 00000100:00100000:2.0:1789760744.801846:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699456 00000100:00100000:2.0:1789760744.801932:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723182965031 time 723226057447 state 0x0 ref 1 00000100:00100000:2.0:1789760744.801970:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723226317863 time 723226057447 state 0x1 ref 1 00000100:00100000:2.0:1789760744.802058:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723226317863 time 723267346194 state 0x0 ref 1 00000100:00100000:2.0:1789760744.802073:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515699456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.802140:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22417408, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.802152:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.802174:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.802179:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.802188:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.802205:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760744.847736:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22417408 -> 22421504 00000001:00080000:1.0:1789760744.847738:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972790 is committed 00010000:00010000:3.0:1789760744.847759:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.847775:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.847785:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.847794:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760744.847800:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.847809:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.847831:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.847838:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1e811 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.847970:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515699456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45880us (46395us total) trans 4294972790 rc 0/0 00000100:00100000:3.0:1789760744.847987:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5502 00000100:00100000:3.0:1789760744.848001:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723226317863 time 723267346194 state 0x2 ref 0 00000100:00100000:0.0:1789760744.850486:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760744.850657:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699584 00000100:00100000:1.0:1789760744.850670:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699584 00000100:00100000:1.0:1789760744.850828:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723226317863 time 723267346194 state 0x0 ref 1 00000100:00100000:1.0:1789760744.850845:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723267606610 time 723267346194 state 0x1 ref 1 00000100:00100000:2.0:1789760744.850998:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723267606610 time 723316283256 state 0x0 ref 1 00000100:00100000:2.0:1789760744.851016:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515699584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760744.851079:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22421504, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xb007 00010000:00010000:2.0:1789760744.851089:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.851108:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.851114:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.851121:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.851137:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.918837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972791 is committed 00002000:00010000:3.0:1789760744.920262:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22421504 -> 22425600 00002000:00010000:3.0:1789760744.920272:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760744 -> 1789760745 00002000:00010000:3.0:1789760744.920274:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760744 -> 1789760745 00010000:00010000:3.0:1789760744.920292:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.920307:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.920316:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.920324:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760744.920339:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.920350:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.920373:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.920381:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1e818 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.920519:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515699584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69501us (70043us total) trans 4294972791 rc 0/0 00000100:00100000:3.0:1789760744.920531:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5503 00000100:00100000:3.0:1789760744.920544:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723267606610 time 723316283256 state 0x2 ref 0 00000100:00100000:2.0:1789760744.923762:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760744.924923:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699712 00000100:00100000:3.0:1789760744.924938:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699712 00000100:00100000:3.0:1789760744.925417:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723267606610 time 723316283256 state 0x0 ref 1 00000100:00100000:3.0:1789760744.925438:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723316543672 time 723316283256 state 0x1 ref 1 00000100:00100000:3.0:1789760744.925488:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723316543672 time 723390776149 state 0x0 ref 1 00000100:00100000:3.0:1789760744.925503:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515699712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760744.925569:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22425600, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:3.0:1789760744.925579:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.925614:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.925619:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.925626:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.925642:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.984160:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972792 is committed 00002000:00010000:3.0:1789760744.984451:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22425600 -> 22429696 00010000:00010000:3.0:1789760744.984470:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.984485:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.984494:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.984501:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760744.984507:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.984516:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.984542:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.984551:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e81f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.984762:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515699712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59209us (60961us total) trans 4294972792 rc 0/0 00000100:00100000:3.0:1789760744.984780:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5504 00000100:00100000:3.0:1789760744.984792:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723316543672 time 723390776149 state 0x2 ref 0 00000100:00100000:1.0:1789760744.987930:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760744.988005:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699840 00000100:00100000:0.0:1789760744.988019:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699840 00000100:00100000:0.0:1789760744.988093:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723316543672 time 723390776149 state 0x0 ref 1 00000100:00100000:0.0:1789760744.988111:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723391036565 time 723390776149 state 0x1 ref 1 00000100:00100000:0.0:1789760744.988184:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723391036565 time 723453471551 state 0x0 ref 1 00000100:00100000:0.0:1789760744.988200:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515699840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760744.988273:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22429696, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:0.0:1789760744.988284:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.988304:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.988310:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.988318:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.988340:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.014749:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972793 is committed 00002000:00010000:3.0:1789760745.014968:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22429696 -> 22433792 00010000:00010000:3.0:1789760745.014990:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.015275:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.015288:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.015297:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.015303:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.015312:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.015339:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.015347:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e826 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.015532:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515699840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27328us (27609us total) trans 4294972793 rc 0/0 00000100:00100000:3.0:1789760745.015545:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5505 00000100:00100000:3.0:1789760745.015558:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723391036565 time 723453471551 state 0x2 ref 0 00000100:00100000:1.0:1789760745.020254:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760745.020372:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515699968 00000100:00100000:0.0:1789760745.020384:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515699968 00000100:00100000:0.0:1789760745.020454:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723391036565 time 723453471551 state 0x0 ref 1 00000100:00100000:0.0:1789760745.020473:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723453731967 time 723453471551 state 0x1 ref 1 00000100:00100000:0.0:1789760745.020538:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723453731967 time 723485826529 state 0x0 ref 1 00000100:00100000:0.0:1789760745.020553:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515699968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760745.020628:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22433792, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:0.0:1789760745.020640:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.020660:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760745.020665:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.020673:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.020690:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.052164:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972794 is committed 00002000:00010000:0.0:1789760745.052269:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22433792 -> 22437888 00010000:00010000:0.0:1789760745.052290:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.052305:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.052315:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.052323:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760745.052329:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.052340:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.052362:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.052371:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e82d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.052513:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515699968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31957us (32267us total) trans 4294972794 rc 0/0 00000100:00100000:0.0:1789760745.052524:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5506 00000100:00100000:0.0:1789760745.052536:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723453731967 time 723485826529 state 0x2 ref 0 00000100:00100000:0.0:1789760745.056116:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760745.056222:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700096 00000100:00100000:1.0:1789760745.056236:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700096 00000100:00100000:1.0:1789760745.056310:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723453731967 time 723485826529 state 0x0 ref 1 00000100:00100000:1.0:1789760745.056329:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723486086945 time 723485826529 state 0x1 ref 1 00000100:00100000:1.0:1789760745.056406:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723486086945 time 723521693804 state 0x0 ref 1 00000100:00100000:1.0:1789760745.056421:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515700096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760745.056489:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22437888, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:1.0:1789760745.056502:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.056580:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760745.056586:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.056594:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.056620:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.094775:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22437888 -> 22441984 00010000:00010000:3.0:1789760745.094805:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.094821:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.094831:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.094839:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.094845:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.094855:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.094891:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.094907:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1e834 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.095038:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515700096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38612us (38929us total) trans 4294972795 rc 0/0 00000100:00100000:3.0:1789760745.095050:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5507 00000100:00100000:3.0:1789760745.095062:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723486086945 time 723521693804 state 0x2 ref 0 00000001:00080000:2.0:1789760745.095493:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972795 is committed 00000100:00100000:1.0:1789760745.098369:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760745.098462:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700224 00000100:00100000:3.0:1789760745.098474:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700224 00000100:00100000:3.0:1789760745.098549:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723486086945 time 723521693804 state 0x0 ref 1 00000100:00100000:3.0:1789760745.098568:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723521954220 time 723521693804 state 0x1 ref 1 00000100:00100000:3.0:1789760745.098616:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723521954220 time 723563904107 state 0x0 ref 1 00000100:00100000:3.0:1789760745.098631:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760745.098696:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22441984, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:3.0:1789760745.098739:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.098762:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760745.098768:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.098777:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.098802:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.129849:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22441984 -> 22446080 00010000:00010000:3.0:1789760745.129938:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.129963:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.129973:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.129980:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.129985:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.129994:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.130019:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.130027:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e83b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.130155:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515700224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31520us (31795us total) trans 4294972796 rc 0/0 00000100:00100000:3.0:1789760745.130168:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5508 00000100:00100000:3.0:1789760745.130180:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723521954220 time 723563904107 state 0x2 ref 0 00000001:00080000:3.0:1789760745.130229:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972796 is committed 00000100:00100000:0.0:1789760745.133560:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760745.133682:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700352 00000100:00100000:3.0:1789760745.133697:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700352 00000100:00100000:3.0:1789760745.133799:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723521954220 time 723563904107 state 0x0 ref 1 00000100:00100000:3.0:1789760745.133820:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723564164523 time 723563904107 state 0x1 ref 1 00000100:00100000:3.0:1789760745.133854:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723564164523 time 723599143494 state 0x0 ref 1 00000100:00100000:3.0:1789760745.133867:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515700352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760745.133947:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22446080, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:3.0:1789760745.133968:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.133989:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760745.133995:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.134002:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.134019:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.187796:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972797 is committed 00002000:00010000:3.0:1789760745.187938:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22446080 -> 22450176 00010000:00010000:3.0:1789760745.187967:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.187982:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.187992:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.188114:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.188121:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.188132:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.188158:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.188166:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e842 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.188288:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515700352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54414us (54734us total) trans 4294972797 rc 0/0 00000100:00100000:3.0:1789760745.188305:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5509 00000100:00100000:3.0:1789760745.188318:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723564164523 time 723599143494 state 0x2 ref 0 00000100:00100000:1.0:1789760745.191376:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760745.191490:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700480 00000100:00100000:3.0:1789760745.191503:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700480 00000100:00100000:3.0:1789760745.191575:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723564164523 time 723599143494 state 0x0 ref 1 00000100:00100000:3.0:1789760745.191594:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723599403910 time 723599143494 state 0x1 ref 1 00000100:00100000:3.0:1789760745.191629:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723599403910 time 723656918239 state 0x0 ref 1 00000100:00100000:3.0:1789760745.191643:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760745.191831:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22450176, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:3.0:1789760745.191844:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.191863:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760745.191868:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.191875:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.191979:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.265766:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972798 is committed 00002000:00010000:3.0:1789760745.266159:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22450176 -> 22454272 00010000:00010000:3.0:1789760745.266181:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.266197:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.266206:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.266214:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.266220:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.266240:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.266265:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.266273:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e849 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.266394:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74745us (75024us total) trans 4294972798 rc 0/0 00000100:00100000:3.0:1789760745.266406:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5510 00000100:00100000:3.0:1789760745.266420:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723599403910 time 723656918239 state 0x2 ref 0 00000100:00100000:1.0:1789760745.273928:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760745.274119:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700608 00000100:00100000:0.0:1789760745.274133:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700608 00000100:00100000:0.0:1789760745.274215:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723599403910 time 723656918239 state 0x0 ref 1 00000100:00100000:0.0:1789760745.274235:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723657178655 time 723656918239 state 0x1 ref 1 00000100:00100000:0.0:1789760745.275236:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723657178655 time 723740517691 state 0x0 ref 1 00000100:00100000:0.0:1789760745.275269:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760745.275365:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22454272, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:0.0:1789760745.275378:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.275399:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760745.275404:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.275412:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.275430:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.318399:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972799 is committed 00002000:00010000:3.0:1789760745.318696:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22454272 -> 22458368 00010000:00010000:3.0:1789760745.318760:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.318775:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.318785:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.318794:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.318800:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.318809:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.318834:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.318842:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e850 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.318987:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43727us (45075us total) trans 4294972799 rc 0/0 00000100:00100000:3.0:1789760745.319000:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5511 00000100:00100000:3.0:1789760745.319012:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723657178655 time 723740517691 state 0x2 ref 0 00000100:00100000:0.0:1789760745.322539:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760745.322670:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700736 00000100:00100000:1.0:1789760745.322686:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700736 00000100:00100000:1.0:1789760745.322802:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723657178655 time 723740517691 state 0x0 ref 1 00000100:00100000:1.0:1789760745.322821:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723740778107 time 723740517691 state 0x1 ref 1 00000100:00100000:1.0:1789760745.322848:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723740778107 time 723788137477 state 0x0 ref 1 00000100:00100000:1.0:1789760745.322861:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760745.322939:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22458368, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:1.0:1789760745.322966:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.322993:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760745.322997:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.323004:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.323022:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.400757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972800 is committed 00002000:00010000:1.0:1789760745.401074:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22458368 -> 22462464 00010000:00010000:1.0:1789760745.401095:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.401111:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.401122:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.401130:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760745.401136:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.401148:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.401172:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.401181:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e857 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.401414:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 78547us (78883us total) trans 4294972800 rc 0/0 00000100:00100000:1.0:1789760745.401427:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5512 00000100:00100000:1.0:1789760745.401440:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723740778107 time 723788137477 state 0x2 ref 0 00000100:00100000:1.0:1789760745.408144:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760745.408245:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700864 00000100:00100000:0.0:1789760745.408258:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700864 00000100:00100000:0.0:1789760745.408334:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723740778107 time 723788137477 state 0x0 ref 1 00000100:00100000:0.0:1789760745.408353:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723788397893 time 723788137477 state 0x1 ref 1 00000100:00100000:0.0:1789760745.408389:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723788397893 time 723873678298 state 0x0 ref 1 00000100:00100000:0.0:1789760745.408404:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515700864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760745.408467:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22462464, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:0.0:1789760745.408478:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.408499:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760745.408505:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.408513:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.408531:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.450516:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22462464 -> 22466560 00010000:00010000:0.0:1789760745.450541:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.450557:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.450575:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.450583:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760745.450589:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.450610:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.450635:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.450645:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e85e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.450794:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515700864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42386us (42658us total) trans 4294972801 rc 0/0 00000100:00100000:0.0:1789760745.450806:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5513 00000100:00100000:0.0:1789760745.450819:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723788397893 time 723873678298 state 0x2 ref 0 00000001:00080000:2.0:1789760745.452212:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972801 is committed 00000100:00100000:2.0:1789760745.456036:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760745.456144:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515700992 00000100:00100000:1.0:1789760745.456158:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515700992 00000100:00100000:1.0:1789760745.456234:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723788397893 time 723873678298 state 0x0 ref 1 00000100:00100000:1.0:1789760745.456255:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723873938714 time 723873678298 state 0x1 ref 1 00000100:00100000:1.0:1789760745.456329:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723873938714 time 723921617438 state 0x0 ref 1 00000100:00100000:1.0:1789760745.456343:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515700992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760745.456408:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22466560, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:1.0:1789760745.456418:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.456438:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760745.456443:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.456451:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.456469:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.503426:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972802 is committed 00002000:00010000:3.0:1789760745.503689:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22466560 -> 22470656 00010000:00010000:3.0:1789760745.504404:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.504423:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.504434:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.504443:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.504449:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.504461:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.504500:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.504508:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e865 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.504633:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515700992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48284us (48603us total) trans 4294972802 rc 0/0 00000100:00100000:3.0:1789760745.504648:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5514 00000100:00100000:3.0:1789760745.504660:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723873938714 time 723921617438 state 0x2 ref 0 00000100:00100000:0.0:1789760745.508865:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760745.509625:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701120 00000100:00100000:3.0:1789760745.509827:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701120 00000100:00100000:3.0:1789760745.510000:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723873938714 time 723921617438 state 0x0 ref 1 00000100:00100000:3.0:1789760745.510021:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723921877854 time 723921617438 state 0x1 ref 1 00000100:00100000:1.0:1789760745.510122:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723921877854 time 723975406991 state 0x0 ref 1 00000100:00100000:1.0:1789760745.510141:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760745.510209:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22470656, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:1.0:1789760745.510219:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.510238:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760745.510243:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.510251:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.510267:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.556746:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972803 is committed 00002000:00010000:2.0:1789760745.558523:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22470656 -> 22474752 00010000:00010000:2.0:1789760745.558552:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.558569:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.558579:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.558587:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760745.558593:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.558603:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.558632:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.558641:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e86c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.558832:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48689us (49972us total) trans 4294972803 rc 0/0 00000100:00100000:2.0:1789760745.558848:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5515 00000100:00100000:2.0:1789760745.558861:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723921877854 time 723975406991 state 0x2 ref 0 00000100:00100000:0.0:1789760745.562502:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760745.562591:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701248 00000100:00100000:3.0:1789760745.562603:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701248 00000100:00100000:3.0:1789760745.562668:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723921877854 time 723975406991 state 0x0 ref 1 00000100:00100000:3.0:1789760745.562685:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723975667407 time 723975406991 state 0x1 ref 1 00000100:00100000:3.0:1789760745.562757:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723975667407 time 724028045473 state 0x0 ref 1 00000100:00100000:3.0:1789760745.562770:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760745.562828:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22474752, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:3.0:1789760745.562838:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.562857:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.562862:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.562878:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.562902:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.606427:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972804 is committed 00002000:00010000:3.0:1789760745.606694:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22474752 -> 22478848 00010000:00010000:3.0:1789760745.606736:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.606751:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.606761:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.606768:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.606774:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.606784:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.606807:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.606816:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e873 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.607104:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44327us (44607us total) trans 4294972804 rc 0/0 00000100:00100000:3.0:1789760745.607116:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5516 00000100:00100000:3.0:1789760745.607128:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723975667407 time 724028045473 state 0x2 ref 0 00000100:00100000:3.0:1789760745.617266:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760745.617470:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701376 00000100:00100000:0.0:1789760745.617570:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701376 00000100:00100000:0.0:1789760745.617647:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 723975667407 time 724028045473 state 0x0 ref 1 00000100:00100000:0.0:1789760745.617668:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724028305889 time 724028045473 state 0x1 ref 1 00000100:00100000:3.0:1789760745.617865:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724028305889 time 724083150933 state 0x0 ref 1 00000100:00100000:2.0:1789760745.618050:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515701376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760745.618121:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22478848, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:2.0:1789760745.618131:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.618152:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.618157:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.618164:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.618180:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.687313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972805 is committed 00002000:00010000:1.0:1789760745.687454:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22478848 -> 22482944 00010000:00010000:1.0:1789760745.687475:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.687490:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.687517:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.687527:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760745.687533:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.687544:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.687570:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.687580:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1e87a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.687699:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515701376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69813us (70442us total) trans 4294972805 rc 0/0 00000100:00100000:1.0:1789760745.687753:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5517 00000100:00100000:1.0:1789760745.687766:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724028305889 time 724083150933 state 0x2 ref 0 00000100:00100000:0.0:1789760745.694889:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760745.695234:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701504 00000100:00100000:2.0:1789760745.695249:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701504 00000100:00100000:2.0:1789760745.695323:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724028305889 time 724083150933 state 0x0 ref 1 00000100:00100000:2.0:1789760745.695344:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724083411349 time 724083150933 state 0x1 ref 1 00000100:00100000:2.0:1789760745.695382:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724083411349 time 724160670775 state 0x0 ref 1 00000100:00100000:2.0:1789760745.695398:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515701504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760745.695471:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22482944, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:2.0:1789760745.695481:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.695501:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.695578:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.695587:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.695605:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.753484:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972806 is committed 00002000:00010000:0.0:1789760745.753650:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22482944 -> 22487040 00010000:00010000:0.0:1789760745.753671:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.753686:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.753696:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.753704:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760745.753737:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.753748:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.753772:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.753780:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1e881 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.754007:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515701504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58605us (59126us total) trans 4294972806 rc 0/0 00000100:00100000:0.0:1789760745.754020:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5518 00000100:00100000:0.0:1789760745.754032:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724083411349 time 724160670775 state 0x2 ref 0 00000100:00100000:3.0:1789760745.757321:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760745.757415:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701632 00000100:00100000:0.0:1789760745.757428:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701632 00000100:00100000:0.0:1789760745.757497:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724083411349 time 724160670775 state 0x0 ref 1 00000100:00100000:0.0:1789760745.757526:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724160931191 time 724160670775 state 0x1 ref 1 00000100:00100000:0.0:1789760745.757558:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724160931191 time 724222846512 state 0x0 ref 1 00000100:00100000:0.0:1789760745.757571:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760745.757648:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22487040, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:0.0:1789760745.757659:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.757680:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.757686:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.757694:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.757746:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.790116:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760745.806181:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972807 is committed 00002000:00010000:0.0:1789760745.806311:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22487040 -> 22491136 00010000:00010000:0.0:1789760745.806332:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.806348:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.806359:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.806368:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760745.806374:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.806383:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.806408:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.806417:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e888 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.806766:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49188us (49452us total) trans 4294972807 rc 0/0 00000100:00100000:0.0:1789760745.806779:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5519 00000100:00100000:0.0:1789760745.806792:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724160931191 time 724222846512 state 0x2 ref 0 00000100:00100000:3.0:1789760745.817487:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760745.817784:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701760 00000100:00100000:0.0:1789760745.817799:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701760 00000100:00100000:0.0:1789760745.817865:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724160931191 time 724222846512 state 0x0 ref 1 00000100:00100000:0.0:1789760745.817885:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724223106928 time 724222846512 state 0x1 ref 1 00000100:00100000:0.0:1789760745.817982:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724223106928 time 724283270222 state 0x0 ref 1 00000100:00100000:0.0:1789760745.817997:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515701760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760745.818082:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22491136, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xb007 00010000:00010000:0.0:1789760745.818093:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.818113:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.818118:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.818127:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.818144:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.875079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972808 is committed 00002000:00010000:3.0:1789760745.875205:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22491136 -> 22495232 00002000:00010000:3.0:1789760745.875222:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760745 -> 1789760746 00002000:00010000:3.0:1789760745.875225:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760745 -> 1789760746 00010000:00010000:3.0:1789760745.875242:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.875258:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.875269:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.875277:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.875283:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.875292:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.875319:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.875328:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e88f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.875445:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515701760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57444us (57966us total) trans 4294972808 rc 0/0 00000100:00100000:3.0:1789760745.875458:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5520 00000100:00100000:3.0:1789760745.875470:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724223106928 time 724283270222 state 0x2 ref 0 00000100:00100000:1.0:1789760745.878844:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760745.878973:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515701888 00000100:00100000:2.0:1789760745.878986:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515701888 00000100:00100000:2.0:1789760745.879059:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724223106928 time 724283270222 state 0x0 ref 1 00000100:00100000:2.0:1789760745.879079:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724283530638 time 724283270222 state 0x1 ref 1 00000100:00100000:2.0:1789760745.879146:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724283530638 time 724344434937 state 0x0 ref 1 00000100:00100000:2.0:1789760745.879161:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515701888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760745.879227:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22495232, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:2.0:1789760745.879237:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.879257:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.879262:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.879270:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.879288:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760745.918545:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22495232 -> 22499328 00010000:00010000:2.0:1789760745.918572:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.918587:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.918606:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.918614:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760745.918620:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.918631:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.918655:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.918662:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e896 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.918838:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515701888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39672us (40131us total) trans 4294972809 rc 0/0 00000100:00100000:2.0:1789760745.918852:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5521 00000100:00100000:2.0:1789760745.918865:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724283530638 time 724344434937 state 0x2 ref 0 00000001:00080000:1.0:1789760745.919526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972809 is committed 00000100:00100000:0.0:1789760745.922489:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760745.922580:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702016 00000100:00100000:3.0:1789760745.922650:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702016 00000100:00100000:3.0:1789760745.922976:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724283530638 time 724344434937 state 0x0 ref 1 00000100:00100000:3.0:1789760745.922995:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724344695353 time 724344434937 state 0x1 ref 1 00000100:00100000:3.0:1789760745.923037:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724344695353 time 724388325627 state 0x0 ref 1 00000100:00100000:3.0:1789760745.923052:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760745.923117:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22499328, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760745.923128:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.923148:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.923154:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.923161:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.923177:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.956226:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972810 is committed 00002000:00010000:3.0:1789760745.956326:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22499328 -> 22503424 00010000:00010000:3.0:1789760745.956346:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956362:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956373:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956382:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760745.956387:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956399:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956426:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.956435:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e89d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.956550:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33493us (34067us total) trans 4294972810 rc 0/0 00000100:00100000:3.0:1789760745.956562:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5522 00000100:00100000:3.0:1789760745.956575:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724344695353 time 724388325627 state 0x2 ref 0 00000100:00100000:2.0:1789760745.959816:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760745.960309:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702144 00000100:00100000:1.0:1789760745.960324:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702144 00000100:00100000:1.0:1789760745.960398:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724344695353 time 724388325627 state 0x0 ref 1 00000100:00100000:1.0:1789760745.960417:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724388586043 time 724388325627 state 0x1 ref 1 00000100:00100000:1.0:1789760745.960452:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724388586043 time 724425740589 state 0x0 ref 1 00000100:00100000:1.0:1789760745.960467:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515702144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760745.960639:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22503424, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:1.0:1789760745.960651:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.960671:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760745.960676:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.960684:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.960702:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.990351:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972811 is committed 00002000:00010000:1.0:1789760745.990481:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22503424 -> 22507520 00010000:00010000:1.0:1789760745.990503:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.990518:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.990535:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.990543:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760745.990549:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.990560:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.990587:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.990665:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.990828:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515702144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30357us (31020us total) trans 4294972811 rc 0/0 00000100:00100000:1.0:1789760745.990840:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5523 00000100:00100000:1.0:1789760745.990853:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724388586043 time 724425740589 state 0x2 ref 0 00000100:00100000:1.0:1789760745.993969:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760745.994077:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702272 00000100:00100000:2.0:1789760745.994091:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702272 00000100:00100000:2.0:1789760745.994163:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724388586043 time 724425740589 state 0x0 ref 1 00000100:00100000:2.0:1789760745.994182:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724426001005 time 724425740589 state 0x1 ref 1 00000100:00100000:2.0:1789760745.994242:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724426001005 time 724459531202 state 0x0 ref 1 00000100:00100000:2.0:1789760745.994256:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760745.994321:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22507520, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:2.0:1789760745.994332:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.994351:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.994356:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.994364:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.994380:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.026036:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972812 is committed 00002000:00010000:1.0:1789760746.026159:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22507520 -> 22511616 00010000:00010000:1.0:1789760746.026181:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.026203:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.026214:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.026223:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.026228:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.026239:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.026264:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.026274:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1e8ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.026398:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32137us (32445us total) trans 4294972812 rc 0/0 00000100:00100000:1.0:1789760746.026409:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5524 00000100:00100000:1.0:1789760746.026422:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724426001005 time 724459531202 state 0x2 ref 0 00000100:00100000:2.0:1789760746.030168:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760746.030581:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702400 00000100:00100000:1.0:1789760746.033128:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702400 00000100:00100000:1.0:1789760746.033358:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724426001005 time 724459531202 state 0x0 ref 1 00000100:00100000:1.0:1789760746.033382:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724459791618 time 724459531202 state 0x1 ref 1 00000100:00100000:1.0:1789760746.033423:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724459791618 time 724498712033 state 0x0 ref 1 00000100:00100000:1.0:1789760746.033438:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515702400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760746.033508:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22511616, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:1.0:1789760746.033518:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.033538:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760746.033544:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.033551:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.033568:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760746.034876:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760746.071113:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972813 is committed 00002000:00010000:1.0:1789760746.071215:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22511616 -> 22515712 00010000:00010000:1.0:1789760746.071236:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.071252:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.071267:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.071275:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.071281:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.071291:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.071317:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.071324:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.071489:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515702400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38040us (41321us total) trans 4294972813 rc 0/0 00000100:00100000:1.0:1789760746.071501:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5525 00000100:00100000:1.0:1789760746.071513:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724459791618 time 724498712033 state 0x2 ref 0 00000100:00100000:2.0:1789760746.075322:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760746.075529:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702528 00000100:00100000:3.0:1789760746.075542:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702528 00000100:00100000:3.0:1789760746.075680:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724459791618 time 724498712033 state 0x0 ref 1 00000100:00100000:3.0:1789760746.075700:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724498972449 time 724498712033 state 0x1 ref 1 00000100:00100000:3.0:1789760746.075918:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724498972449 time 724541205824 state 0x0 ref 1 00000100:00100000:3.0:1789760746.075933:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760746.076009:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22515712, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760746.076020:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.076038:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760746.076044:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.076052:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.076067:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.115001:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972814 is committed 00002000:00010000:3.0:1789760746.115246:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22515712 -> 22519808 00010000:00010000:3.0:1789760746.115265:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.115281:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.115292:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.115300:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760746.115307:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.115317:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.115344:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.115353:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.115511:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39575us (40198us total) trans 4294972814 rc 0/0 00000100:00100000:3.0:1789760746.115524:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5526 00000100:00100000:3.0:1789760746.115535:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724498972449 time 724541205824 state 0x2 ref 0 00000100:00100000:3.0:1789760746.118834:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760746.119008:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702656 00000100:00100000:1.0:1789760746.119022:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702656 00000100:00100000:1.0:1789760746.119098:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724498972449 time 724541205824 state 0x0 ref 1 00000100:00100000:1.0:1789760746.119119:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724541466240 time 724541205824 state 0x1 ref 1 00000100:00100000:1.0:1789760746.119160:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724541466240 time 724584449869 state 0x0 ref 1 00000100:00100000:1.0:1789760746.119178:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515702656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760746.119254:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22519808, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:1.0:1789760746.119266:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.119286:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760746.119292:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.119300:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.119316:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.143046:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22519808 -> 22523904 00010000:00010000:1.0:1789760746.143077:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.143095:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.143106:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.143115:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.143121:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.143131:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.143163:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.143170:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e8c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.143294:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515702656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24113us (24471us total) trans 4294972815 rc 0/0 00000100:00100000:1.0:1789760746.143306:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5527 00000100:00100000:1.0:1789760746.143319:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724541466240 time 724584449869 state 0x2 ref 0 00000001:00080000:3.0:1789760746.146189:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972815 is committed 00000100:00100000:3.0:1789760746.147254:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.147367:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702784 00000100:00100000:0.0:1789760746.147381:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702784 00000100:00100000:0.0:1789760746.147455:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724541466240 time 724584449869 state 0x0 ref 1 00000100:00100000:0.0:1789760746.147475:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724584710285 time 724584449869 state 0x1 ref 1 00000100:00100000:0.0:1789760746.147542:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724584710285 time 724612829288 state 0x0 ref 1 00000100:00100000:0.0:1789760746.147557:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.147633:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22523904, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:0.0:1789760746.147644:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.147663:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760746.147668:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.147676:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.147694:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.182913:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972816 is committed 00002000:00010000:0.0:1789760746.183108:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22523904 -> 22528000 00010000:00010000:0.0:1789760746.183129:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.183145:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.183160:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.183169:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760746.183175:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.183185:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.183215:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.183224:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.183350:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515702784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35789us (36100us total) trans 4294972816 rc 0/0 00000100:00100000:0.0:1789760746.183364:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5528 00000100:00100000:0.0:1789760746.183376:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724584710285 time 724612829288 state 0x2 ref 0 00000100:00100000:1.0:1789760746.186383:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760746.186537:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515702912 00000100:00100000:3.0:1789760746.186549:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515702912 00000100:00100000:3.0:1789760746.186688:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724584710285 time 724612829288 state 0x0 ref 1 00000100:00100000:3.0:1789760746.186815:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724613089704 time 724612829288 state 0x1 ref 1 00000100:00100000:3.0:1789760746.186879:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724613089704 time 724652167945 state 0x0 ref 1 00000100:00100000:3.0:1789760746.186895:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515702912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760746.186968:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22528000, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760746.186978:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.186997:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760746.187002:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.187010:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.187030:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.258102:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972817 is committed 00002000:00010000:3.0:1789760746.258647:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22528000 -> 22532096 00010000:00010000:3.0:1789760746.258668:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.258682:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.258692:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.258699:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760746.258736:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.258747:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.258773:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.258782:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.259061:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515702912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 72162us (72685us total) trans 4294972817 rc 0/0 00000100:00100000:3.0:1789760746.259074:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5529 00000100:00100000:3.0:1789760746.259086:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724613089704 time 724652167945 state 0x2 ref 0 00000100:00100000:1.0:1789760746.262872:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760746.263032:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703040 00000100:00100000:2.0:1789760746.263045:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703040 00000100:00100000:2.0:1789760746.263118:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724613089704 time 724652167945 state 0x0 ref 1 00000100:00100000:2.0:1789760746.263137:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724652428361 time 724652167945 state 0x1 ref 1 00000100:00100000:2.0:1789760746.263172:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724652428361 time 724728461144 state 0x0 ref 1 00000100:00100000:2.0:1789760746.263187:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515703040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760746.263250:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22532096, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:2.0:1789760746.263261:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.263282:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760746.263288:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.263295:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.263318:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.331991:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972818 is committed 00002000:00010000:1.0:1789760746.332121:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22532096 -> 22536192 00010000:00010000:1.0:1789760746.332142:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.332158:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.332168:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.332178:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.332184:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.332195:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.332221:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.332228:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e8d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.332352:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515703040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69160us (69489us total) trans 4294972818 rc 0/0 00000100:00100000:1.0:1789760746.332365:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5530 00000100:00100000:1.0:1789760746.332377:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724652428361 time 724728461144 state 0x2 ref 0 00000100:00100000:1.0:1789760746.335833:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.335993:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703168 00000100:00100000:0.0:1789760746.336006:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703168 00000100:00100000:0.0:1789760746.336078:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724652428361 time 724728461144 state 0x0 ref 1 00000100:00100000:0.0:1789760746.336098:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724728721560 time 724728461144 state 0x1 ref 1 00000100:00100000:0.0:1789760746.336168:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724728721560 time 724801456917 state 0x0 ref 1 00000100:00100000:0.0:1789760746.336183:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515703168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.336246:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22536192, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:0.0:1789760746.336257:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.336276:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760746.336282:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.336289:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.336307:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.379793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972819 is committed 00002000:00010000:3.0:1789760746.380124:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22536192 -> 22540288 00010000:00010000:3.0:1789760746.380146:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.380161:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.380172:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.380181:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760746.380188:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.380197:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.380224:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.380239:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1e8dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.380367:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515703168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44179us (44541us total) trans 4294972819 rc 0/0 00000100:00100000:3.0:1789760746.380380:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5531 00000100:00100000:3.0:1789760746.380391:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724728721560 time 724801456917 state 0x2 ref 0 00000100:00100000:1.0:1789760746.383564:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.383897:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703296 00000100:00100000:0.0:1789760746.383910:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703296 00000100:00100000:0.0:1789760746.383989:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724728721560 time 724801456917 state 0x0 ref 1 00000100:00100000:0.0:1789760746.384009:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724801717333 time 724801456917 state 0x1 ref 1 00000100:00100000:0.0:1789760746.384039:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724801717333 time 724849327182 state 0x0 ref 1 00000100:00100000:0.0:1789760746.384053:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515703296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.384121:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22540288, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:0.0:1789760746.384132:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.384151:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760746.384156:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.384163:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.384180:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.416324:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972820 is committed 00002000:00010000:0.0:1789760746.416434:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22540288 -> 22544384 00010000:00010000:0.0:1789760746.416455:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.416470:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.416481:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.416490:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760746.416496:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.416506:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.416532:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.416540:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e8e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.416819:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515703296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32760us (33264us total) trans 4294972820 rc 0/0 00000100:00100000:0.0:1789760746.416831:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5532 00000100:00100000:0.0:1789760746.416844:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724801717333 time 724849327182 state 0x2 ref 0 00000100:00100000:0.0:1789760746.420694:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760746.420971:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703424 00000100:00100000:1.0:1789760746.420985:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703424 00000100:00100000:1.0:1789760746.421059:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724801717333 time 724849327182 state 0x0 ref 1 00000100:00100000:1.0:1789760746.421079:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724849587598 time 724849327182 state 0x1 ref 1 00000100:00100000:1.0:1789760746.421177:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724849587598 time 724886464871 state 0x0 ref 1 00000100:00100000:1.0:1789760746.421192:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515703424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760746.421257:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22544384, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:1.0:1789760746.421268:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.421290:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760746.421295:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.421302:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.421318:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.456264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972821 is committed 00002000:00010000:3.0:1789760746.456394:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22544384 -> 22548480 00010000:00010000:3.0:1789760746.456413:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.456430:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.456441:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.456450:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760746.456456:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.456466:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.456496:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.456505:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e8ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.456825:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515703424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35626us (36137us total) trans 4294972821 rc 0/0 00000100:00100000:3.0:1789760746.456839:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5533 00000100:00100000:3.0:1789760746.456852:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724849587598 time 724886464871 state 0x2 ref 0 00000100:00100000:1.0:1789760746.460151:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760746.460235:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703552 00000100:00100000:3.0:1789760746.460247:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703552 00000100:00100000:3.0:1789760746.460313:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724849587598 time 724886464871 state 0x0 ref 1 00000100:00100000:3.0:1789760746.460332:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724886725287 time 724886464871 state 0x1 ref 1 00000100:00100000:3.0:1789760746.460405:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724886725287 time 724925693420 state 0x0 ref 1 00000100:00100000:3.0:1789760746.460420:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515703552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760746.460481:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22548480, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760746.460491:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.460511:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760746.460517:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.460525:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.460542:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.502240:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22548480 -> 22552576 00010000:00010000:0.0:1789760746.502269:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.502285:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.502295:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.502303:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760746.502309:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.502320:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.502344:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.502352:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e8f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.502476:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515703552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42051us (42331us total) trans 4294972822 rc 0/0 00000100:00100000:0.0:1789760746.502488:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5534 00000100:00100000:0.0:1789760746.502506:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724886725287 time 724925693420 state 0x2 ref 0 00000001:00080000:2.0:1789760746.503249:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972822 is committed 00000100:00100000:1.0:1789760746.508670:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760746.508909:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703680 00000100:00100000:3.0:1789760746.508930:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703680 00000100:00100000:3.0:1789760746.509026:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724886725287 time 724925693420 state 0x0 ref 1 00000100:00100000:3.0:1789760746.509044:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724925953836 time 724925693420 state 0x1 ref 1 00000100:00100000:3.0:1789760746.509081:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724925953836 time 724974370167 state 0x0 ref 1 00000100:00100000:3.0:1789760746.509095:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515703680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760746.509429:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22552576, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760746.509444:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.509466:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760746.509471:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.509479:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.509497:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.542439:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972823 is committed 00002000:00010000:2.0:1789760746.542563:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22552576 -> 22556672 00010000:00010000:2.0:1789760746.542585:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.542601:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.542611:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.542619:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760746.542626:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.542637:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.542663:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.542672:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e8f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.542861:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515703680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33760us (34199us total) trans 4294972823 rc 0/0 00000100:00100000:2.0:1789760746.542880:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5535 00000100:00100000:2.0:1789760746.542892:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724925953836 time 724974370167 state 0x2 ref 0 00000100:00100000:1.0:1789760746.547359:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760746.547927:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703808 00000100:00100000:2.0:1789760746.547952:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703808 00000100:00100000:2.0:1789760746.548034:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724925953836 time 724974370167 state 0x0 ref 1 00000100:00100000:2.0:1789760746.548053:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724974630583 time 724974370167 state 0x1 ref 1 00000100:00100000:2.0:1789760746.548137:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724974630583 time 725013424515 state 0x0 ref 1 00000100:00100000:2.0:1789760746.548151:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515703808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760746.548221:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22556672, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:2.0:1789760746.548231:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.548252:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760746.548257:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.548264:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.548280:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.580617:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972824 is committed 00002000:00010000:2.0:1789760746.580762:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22556672 -> 22560768 00010000:00010000:2.0:1789760746.580784:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.580796:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.580804:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.580809:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760746.580814:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.580823:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.580849:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.580857:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1e8ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.581145:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515703808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32989us (33795us total) trans 4294972824 rc 0/0 00000100:00100000:2.0:1789760746.581162:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5536 00000100:00100000:2.0:1789760746.581240:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724974630583 time 725013424515 state 0x2 ref 0 00000100:00100000:1.0:1789760746.587823:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.588100:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515703936 00000100:00100000:0.0:1789760746.588113:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515703936 00000100:00100000:0.0:1789760746.588187:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 724974630583 time 725013424515 state 0x0 ref 1 00000100:00100000:0.0:1789760746.588205:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725013684931 time 725013424515 state 0x1 ref 1 00000100:00100000:0.0:1789760746.588287:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725013684931 time 725053574313 state 0x0 ref 1 00000100:00100000:0.0:1789760746.588302:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515703936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.588364:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22560768, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:0.0:1789760746.588375:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588395:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.588400:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.588408:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588425:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.625739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972825 is committed 00002000:00010000:1.0:1789760746.625853:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22560768 -> 22564864 00010000:00010000:1.0:1789760746.625938:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.625962:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.625978:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.626096:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.626104:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.626115:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.626142:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.626151:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1e906 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.626323:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515703936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38016us (38606us total) trans 4294972825 rc 0/0 00000100:00100000:1.0:1789760746.626335:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5537 00000100:00100000:1.0:1789760746.626348:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725013684931 time 725053574313 state 0x2 ref 0 00000100:00100000:2.0:1789760746.637315:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.638207:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704064 00000100:00100000:0.0:1789760746.638224:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704064 00000100:00100000:0.0:1789760746.638303:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725013684931 time 725053574313 state 0x0 ref 1 00000100:00100000:0.0:1789760746.638323:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725053834729 time 725053574313 state 0x1 ref 1 00000100:00100000:0.0:1789760746.638464:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725053834729 time 725103751230 state 0x0 ref 1 00000100:00100000:0.0:1789760746.638480:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515704064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.638543:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22564864, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:0.0:1789760746.638554:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.638575:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.638581:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.638589:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.638608:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.678421:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972826 is committed 00002000:00010000:0.0:1789760746.678644:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22564864 -> 22568960 00010000:00010000:0.0:1789760746.678665:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.678680:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.678690:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.678698:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760746.678704:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.678739:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.678764:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.678772:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e90d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.678901:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515704064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40416us (41594us total) trans 4294972826 rc 0/0 00000100:00100000:0.0:1789760746.678914:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5538 00000100:00100000:0.0:1789760746.678930:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725053834729 time 725103751230 state 0x2 ref 0 00000100:00100000:0.0:1789760746.684661:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760746.684784:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704192 00000100:00100000:3.0:1789760746.684798:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704192 00000100:00100000:3.0:1789760746.684984:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725053834729 time 725103751230 state 0x0 ref 1 00000100:00100000:3.0:1789760746.685005:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725104011646 time 725103751230 state 0x1 ref 1 00000100:00100000:3.0:1789760746.685043:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725104011646 time 725150330541 state 0x0 ref 1 00000100:00100000:3.0:1789760746.685058:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760746.685127:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22568960, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760746.685139:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.685161:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.685166:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.685174:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.685191:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.783821:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972827 is committed 00002000:00010000:3.0:1789760746.784069:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22568960 -> 22573056 00010000:00010000:3.0:1789760746.784091:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.784114:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.784124:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.784131:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760746.784137:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.784147:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.784172:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.784179:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e914 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.784302:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 99239us (99649us total) trans 4294972827 rc 0/0 00000100:00100000:3.0:1789760746.784315:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5539 00000100:00100000:3.0:1789760746.784327:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725104011646 time 725150330541 state 0x2 ref 0 00000100:00100000:1.0:1789760746.787285:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760746.787380:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704320 00000100:00100000:3.0:1789760746.787394:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704320 00000100:00100000:3.0:1789760746.787465:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725104011646 time 725150330541 state 0x0 ref 1 00000100:00100000:3.0:1789760746.787484:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725150590957 time 725150330541 state 0x1 ref 1 00000100:00100000:3.0:1789760746.787517:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725150590957 time 725252805840 state 0x0 ref 1 00000100:00100000:3.0:1789760746.787531:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515704320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760746.787595:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22573056, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:3.0:1789760746.787606:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.787628:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.787633:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.787642:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.787658:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.812833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972828 is committed 00002000:00010000:0.0:1789760746.813037:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22573056 -> 22577152 00010000:00010000:0.0:1789760746.813059:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.813075:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.813086:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.813094:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760746.813101:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.813112:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.813138:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.813153:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e91b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.813321:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515704320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25784us (26041us total) trans 4294972828 rc 0/0 00000100:00100000:0.0:1789760746.813334:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5540 00000100:00100000:0.0:1789760746.813347:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725150590957 time 725252805840 state 0x2 ref 0 00010000:00010000:0.0:1789760746.813988:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760746.819139:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760746.819357:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704448 00000100:00100000:1.0:1789760746.819371:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704448 00000100:00100000:1.0:1789760746.819456:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725150590957 time 725252805840 state 0x0 ref 1 00000100:00100000:1.0:1789760746.819479:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725253066256 time 725252805840 state 0x1 ref 1 00000100:00100000:1.0:1789760746.819514:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725253066256 time 725284802339 state 0x0 ref 1 00000100:00100000:1.0:1789760746.819669:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760746.819854:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22577152, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xb007 00010000:00010000:1.0:1789760746.819866:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.819887:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760746.819892:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.819899:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.819917:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.860441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972829 is committed 00002000:00010000:3.0:1789760746.860569:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22577152 -> 22581248 00002000:00010000:3.0:1789760746.860574:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760746 -> 1789760747 00002000:00010000:3.0:1789760746.860577:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760746 -> 1789760747 00010000:00010000:3.0:1789760746.860593:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.860607:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.860618:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.860626:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760746.860632:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.860642:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.860666:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.860674:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1e922 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.860825:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41294us (41692us total) trans 4294972829 rc 0/0 00000100:00100000:3.0:1789760746.860837:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5541 00000100:00100000:3.0:1789760746.860851:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725253066256 time 725284802339 state 0x2 ref 0 00000100:00100000:2.0:1789760746.864307:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760746.864439:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704576 00000100:00100000:1.0:1789760746.864454:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704576 00000100:00100000:1.0:1789760746.864528:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725253066256 time 725284802339 state 0x0 ref 1 00000100:00100000:1.0:1789760746.864546:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725285062755 time 725284802339 state 0x1 ref 1 00000100:00100000:1.0:1789760746.864581:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725285062755 time 725329869035 state 0x0 ref 1 00000100:00100000:1.0:1789760746.864598:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515704576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760746.864663:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22581248, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760746.864674:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.864695:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760746.864701:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.864750:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.864768:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.911032:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972830 is committed 00002000:00010000:1.0:1789760746.911278:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22581248 -> 22585344 00010000:00010000:1.0:1789760746.911298:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.911312:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.911322:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.911330:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.911336:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.911346:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.911370:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.911379:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1e929 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.911532:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515704576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46930us (47233us total) trans 4294972830 rc 0/0 00000100:00100000:1.0:1789760746.911545:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5542 00000100:00100000:1.0:1789760746.911557:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725285062755 time 725329869035 state 0x2 ref 0 00000100:00100000:1.0:1789760746.914282:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.914381:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704704 00000100:00100000:0.0:1789760746.914399:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704704 00000100:00100000:0.0:1789760746.914478:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725285062755 time 725329869035 state 0x0 ref 1 00000100:00100000:0.0:1789760746.914497:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725330129451 time 725329869035 state 0x1 ref 1 00000100:00100000:0.0:1789760746.914530:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725330129451 time 725379818975 state 0x0 ref 1 00000100:00100000:0.0:1789760746.914544:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.914667:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22585344, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:0.0:1789760746.914678:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.914699:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.914776:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.914786:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.914804:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.953035:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972831 is committed 00002000:00010000:1.0:1789760746.953164:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22585344 -> 22589440 00010000:00010000:1.0:1789760746.953185:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.953201:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.953211:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.953225:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760746.953232:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.953242:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.953265:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.953273:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1e930 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.953400:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38852us (39123us total) trans 4294972831 rc 0/0 00000100:00100000:1.0:1789760746.953412:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5543 00000100:00100000:1.0:1789760746.953424:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725330129451 time 725379818975 state 0x2 ref 0 00000100:00100000:2.0:1789760746.956824:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760746.957013:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704832 00000100:00100000:0.0:1789760746.957026:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704832 00000100:00100000:0.0:1789760746.957094:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725330129451 time 725379818975 state 0x0 ref 1 00000100:00100000:0.0:1789760746.957112:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725380079391 time 725379818975 state 0x1 ref 1 00000100:00100000:0.0:1789760746.957182:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725380079391 time 725422470936 state 0x0 ref 1 00000100:00100000:0.0:1789760746.957195:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515704832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760746.957256:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22589440, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:0.0:1789760746.957267:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.957286:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.957291:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.957298:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.957314:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.007498:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544412928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.007649:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760747.007902:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544412928 00000100:00100000:0.0:1789760747.007913:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544412928 00000100:00100000:1.0:1789760747.007971:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760747.007978:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544413312:0@lo:400:kworker.0 00000100:00100000:0.0:1789760747.007986:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:0.0:1789760747.007996:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909352850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544412928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.008011:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760747.008047:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413312 00000100:00100000:1.0:1789760747.008051:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413312 00000100:00100000:0.0:1789760747.008057:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909352850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544412928:12345-0@lo:400:kworker.0 Request processed in 63us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760747.008065:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1789760747.008094:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:1.0:1789760747.008101:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544413312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.008137:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544413312:12345-0@lo:400:kworker.0 Request processed in 37us (125us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.008143:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00080000:1.0:1789760747.008162:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760747.008200:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544413312:0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.008283:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544412928:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.008376:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760747.008386:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544413440:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.008524:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760747.008562:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544413056:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.008650:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760747.008703:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544413568:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.008753:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760747.008850:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413440 00000100:00100000:1.0:1789760747.008856:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413440 00000100:00100000:1.0:1789760747.008891:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413056 00000100:00100000:1.0:1789760747.008894:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413056 00000100:00100000:1.0:1789760747.008911:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789760747.008918:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544413440:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.008968:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544413440:12345-0@lo:400:kworker.0 Request processed in 51us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.008975:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1789760747.008987:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:1.0:1789760747.008992:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544413056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.009018:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544413056:12345-0@lo:400:kworker.0 Request processed in 27us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.009024:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00080000:1.0:1789760747.009044:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760747.009064:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544413440:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.009126:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544413056:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.009134:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760747.009140:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544413184:0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.009160:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760747.009192:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413568 00000100:00100000:3.0:1789760747.009196:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413568 00000100:00100000:3.0:1789760747.009295:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:3.0:1789760747.009302:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544413568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760747.009334:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544413568:12345-0@lo:400:kworker.0 Request processed in 33us (580us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.009340:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:3.0:1789760747.009365:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544413568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.009478:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413184 00000100:00100000:1.0:1789760747.009482:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413184 00000100:00100000:1.0:1789760747.009543:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1789760747.009551:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544413184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760747.009680:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544413184:12345-0@lo:400:kworker.0 Request processed in 130us (519us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.009686:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00080000:1.0:1789760747.009701:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760747.009704:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760747.009856:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544413184:0@lo:400:kworker.0 00002000:00010000:2.0:1789760747.020290:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22589440 -> 22593536 00010000:00010000:2.0:1789760747.020311:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.020325:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.020335:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.020343:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.020347:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.020357:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.020377:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.020385:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1e937 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.020611:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515704832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63412us (63797us total) trans 4294972832 rc 0/0 00000100:00100000:2.0:1789760747.020622:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5544 00000100:00100000:2.0:1789760747.020633:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725380079391 time 725422470936 state 0x2 ref 0 00000001:00080000:3.0:1789760747.020748:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972832 is committed 00000100:00100000:2.0:1789760747.028122:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.028217:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515704960 00000100:00100000:3.0:1789760747.028228:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515704960 00000100:00100000:3.0:1789760747.028353:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725380079391 time 725422470936 state 0x0 ref 1 00000100:00100000:3.0:1789760747.028371:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725422731352 time 725422470936 state 0x1 ref 1 00000100:00100000:3.0:1789760747.028465:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725422731352 time 725493752881 state 0x0 ref 1 00000100:00100000:3.0:1789760747.028480:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515704960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760747.028529:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22593536, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:3.0:1789760747.028539:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.028554:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760747.028559:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.028566:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.028650:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.064885:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22593536 -> 22597632 00010000:00010000:3.0:1789760747.064907:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.064921:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.064931:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.064949:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760747.064955:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.064965:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.064988:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.064998:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e93e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.065101:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515704960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36618us (36984us total) trans 4294972833 rc 0/0 00000100:00100000:3.0:1789760747.065111:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5545 00000100:00100000:3.0:1789760747.065121:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725422731352 time 725493752881 state 0x2 ref 0 00000001:00080000:0.0:1789760747.066079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972833 is committed 00000100:00100000:1.0:1789760747.069421:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.069508:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705088 00000100:00100000:3.0:1789760747.069518:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705088 00000100:00100000:3.0:1789760747.069574:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725422731352 time 725493752881 state 0x0 ref 1 00000100:00100000:3.0:1789760747.069590:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725494013297 time 725493752881 state 0x1 ref 1 00000100:00100000:3.0:1789760747.069621:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725494013297 time 725534909837 state 0x0 ref 1 00000100:00100000:3.0:1789760747.069633:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760747.069686:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22597632, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:3.0:1789760747.069695:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.069752:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760747.069758:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.069765:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.069781:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.110112:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972834 is committed 00002000:00010000:3.0:1789760747.110313:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22597632 -> 22601728 00010000:00010000:3.0:1789760747.110328:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.110337:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.110351:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.110356:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760747.110360:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.110366:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.110381:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.110487:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e945 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.110583:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40946us (41167us total) trans 4294972834 rc 0/0 00000100:00100000:3.0:1789760747.110593:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5546 00000100:00100000:3.0:1789760747.110604:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725494013297 time 725534909837 state 0x2 ref 0 00000100:00100000:1.0:1789760747.113519:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.113629:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705216 00000100:00100000:3.0:1789760747.113638:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705216 00000100:00100000:3.0:1789760747.113688:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725494013297 time 725534909837 state 0x0 ref 1 00000100:00100000:3.0:1789760747.113703:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725535170253 time 725534909837 state 0x1 ref 1 00000100:00100000:3.0:1789760747.113752:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725535170253 time 725579041362 state 0x0 ref 1 00000100:00100000:3.0:1789760747.113765:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515705216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760747.113811:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22601728, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:3.0:1789760747.113819:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.113833:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760747.113852:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.113858:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.113884:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.139641:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972835 is committed 00002000:00010000:1.0:1789760747.139778:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22601728 -> 22605824 00010000:00010000:1.0:1789760747.139794:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.139807:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.139815:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.139822:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760747.139827:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.139848:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.139882:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.139891:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1e94c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.140000:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515705216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26232us (26485us total) trans 4294972835 rc 0/0 00000100:00100000:1.0:1789760747.140010:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5547 00000100:00100000:1.0:1789760747.140020:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725535170253 time 725579041362 state 0x2 ref 0 00000100:00100000:2.0:1789760747.143433:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.143515:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705344 00000100:00100000:1.0:1789760747.143525:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705344 00000100:00100000:1.0:1789760747.143573:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725535170253 time 725579041362 state 0x0 ref 1 00000100:00100000:1.0:1789760747.143587:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725579301778 time 725579041362 state 0x1 ref 1 00000100:00100000:1.0:1789760747.143610:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725579301778 time 725608899312 state 0x0 ref 1 00000100:00100000:1.0:1789760747.143620:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.143667:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22605824, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760747.143676:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.143694:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760747.143699:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.143749:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.143764:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.173207:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972836 is committed 00002000:00010000:2.0:1789760747.173314:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22605824 -> 22609920 00010000:00010000:2.0:1789760747.173330:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173343:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173352:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173360:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.173365:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173374:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173403:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.173412:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1e953 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.173506:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29882us (30078us total) trans 4294972836 rc 0/0 00000100:00100000:2.0:1789760747.173517:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5548 00000100:00100000:2.0:1789760747.173527:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725579301778 time 725608899312 state 0x2 ref 0 00000100:00100000:2.0:1789760747.180197:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.183118:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705472 00000100:00100000:1.0:1789760747.183133:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705472 00000100:00100000:0.0:1789760747.184531:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725579301778 time 725608899312 state 0x0 ref 1 00000100:00100000:0.0:1789760747.184548:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725609159728 time 725608899312 state 0x1 ref 1 00000100:00100000:0.0:1789760747.184585:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725609159728 time 725649874531 state 0x0 ref 1 00000100:00100000:0.0:1789760747.184600:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515705472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760747.184650:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22609920, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:0.0:1789760747.184658:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.184674:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760747.184678:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.184684:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.184698:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.216330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972837 is committed 00002000:00010000:1.0:1789760747.216439:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22609920 -> 22614016 00010000:00010000:1.0:1789760747.216455:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.216466:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.216474:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.216481:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760747.216485:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.216494:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.216513:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.216525:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1e95a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.216745:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515705472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32105us (36516us total) trans 4294972837 rc 0/0 00000100:00100000:1.0:1789760747.216755:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5549 00000100:00100000:1.0:1789760747.216765:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725609159728 time 725649874531 state 0x2 ref 0 00000100:00100000:1.0:1789760747.225979:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760747.226082:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705600 00000100:00100000:2.0:1789760747.226093:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705600 00000100:00100000:2.0:1789760747.226160:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725609159728 time 725649874531 state 0x0 ref 1 00000100:00100000:2.0:1789760747.226176:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725650134947 time 725649874531 state 0x1 ref 1 00000100:00100000:2.0:1789760747.226208:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725650134947 time 725691497558 state 0x0 ref 1 00000100:00100000:2.0:1789760747.226222:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.226269:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22614016, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:2.0:1789760747.226278:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.226293:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760747.226298:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.226304:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.226317:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.298561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972838 is committed 00002000:00010000:2.0:1789760747.298665:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22614016 -> 22618112 00010000:00010000:2.0:1789760747.298681:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.298695:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.298704:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.298738:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.298743:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.298751:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.298770:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.298778:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e961 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.298986:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 72753us (73003us total) trans 4294972838 rc 0/0 00000100:00100000:2.0:1789760747.298997:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5550 00000100:00100000:2.0:1789760747.299007:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725650134947 time 725691497558 state 0x2 ref 0 00000100:00100000:2.0:1789760747.308677:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.308956:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705728 00000100:00100000:3.0:1789760747.308966:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705728 00000100:00100000:2.0:1789760747.309208:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725650134947 time 725691497558 state 0x0 ref 1 00000100:00100000:2.0:1789760747.309223:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725691757974 time 725691497558 state 0x1 ref 1 00000100:00100000:2.0:1789760747.309354:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725691757974 time 725774641840 state 0x0 ref 1 00000100:00100000:2.0:1789760747.309367:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515705728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.309416:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22618112, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:2.0:1789760747.309424:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.309439:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760747.309444:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.309461:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.309475:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.384027:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22618112 -> 22622208 00010000:00010000:3.0:1789760747.384057:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.384073:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.384084:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.384094:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760747.384099:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.384110:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.384136:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.384150:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1e968 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.384267:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515705728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74895us (75685us total) trans 4294972839 rc 0/0 00000100:00100000:3.0:1789760747.384280:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5551 00000100:00100000:3.0:1789760747.384292:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725691757974 time 725774641840 state 0x2 ref 0 00000001:00080000:1.0:1789760747.389095:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972839 is committed 00000100:00100000:0.0:1789760747.391437:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.392399:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705856 00000100:00100000:3.0:1789760747.392418:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705856 00000100:00100000:2.0:1789760747.394319:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725691757974 time 725774641840 state 0x0 ref 1 00000100:00100000:2.0:1789760747.394345:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725774902256 time 725774641840 state 0x1 ref 1 00000100:00100000:2.0:1789760747.394403:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725774902256 time 725859691010 state 0x0 ref 1 00000100:00100000:2.0:1789760747.394429:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515705856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.394503:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22622208, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:2.0:1789760747.394516:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.394537:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760747.394542:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.394550:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.394568:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.440995:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972840 is committed 00000100:00100000:0.0:1789760747.441230:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760747.441501:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760747.441544:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.441595:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515705984 00000100:00100000:3.0:1789760747.441608:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515705984 00000100:00100000:3.0:1789760747.441675:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 394 00000100:00100000:3.0:1789760747.441688:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909350050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515705984:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760747.441815:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909350050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515705984:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 130us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.441825:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 394 00002000:00010000:2.0:1789760747.442794:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22622208 -> 22626304 00010000:00010000:2.0:1789760747.442818:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.442833:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.442844:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.442853:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.442870:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.442881:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.442906:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.442915:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1e96f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.443021:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515705856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48602us (51596us total) trans 4294972840 rc 0/0 00000100:00100000:2.0:1789760747.443033:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5552 00000100:00100000:2.0:1789760747.443045:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725774902256 time 725859691010 state 0x2 ref 0 00000100:00100000:1.0:1789760747.443970:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706112 00000100:00100000:1.0:1789760747.443982:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706112 00000100:00100000:1.0:1789760747.444094:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706240 00000100:00100000:1.0:1789760747.444099:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706240 00000100:00080000:3.0:1789760747.444144:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760747.444183:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 461 00000100:00100000:0.0:1789760747.444197:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515706112:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760747.444495:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 462 00000100:00100000:2.0:1789760747.444507:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515706240:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760747.444553:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f436680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515706112:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 359us (3329us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760747.444561:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 461 00000100:00100000:2.0:1789760747.444703:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515706240:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 199us (3157us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760747.444784:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 462 00000100:00100000:3.0:1789760747.452631:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.452742:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706368 00000100:00100000:1.0:1789760747.452752:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706368 00000100:00100000:1.0:1789760747.452806:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725774902256 time 725859691010 state 0x0 ref 1 00000100:00100000:1.0:1789760747.452822:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725859951426 time 725859691010 state 0x1 ref 1 00000100:00100000:1.0:1789760747.452871:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725859951426 time 725918160020 state 0x0 ref 1 00000100:00100000:1.0:1789760747.452885:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515706368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.452949:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22626304, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760747.452959:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.452980:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760747.452984:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.452991:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.453004:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.495442:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972841 is committed 00002000:00010000:1.0:1789760747.495603:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22626304 -> 22630400 00010000:00010000:1.0:1789760747.495619:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.495633:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.495643:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.495651:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760747.495656:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.495665:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.495683:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.495691:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e976 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.495805:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515706368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42917us (43178us total) trans 4294972841 rc 0/0 00000100:00100000:1.0:1789760747.495816:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5553 00000100:00100000:1.0:1789760747.495825:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725859951426 time 725918160020 state 0x2 ref 0 00000100:00100000:3.0:1789760747.509284:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.509509:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706496 00000100:00100000:1.0:1789760747.509520:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706496 00000100:00100000:1.0:1789760747.509578:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725859951426 time 725918160020 state 0x0 ref 1 00000100:00100000:1.0:1789760747.509609:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725918420436 time 725918160020 state 0x1 ref 1 00000100:00100000:1.0:1789760747.509643:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725918420436 time 725974931968 state 0x0 ref 1 00000100:00100000:1.0:1789760747.509656:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cabc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515706496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.509750:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22630400, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760747.509760:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.509775:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760747.509779:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.509786:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.509804:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760747.546178:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22630400 -> 22634496 00010000:00010000:1.0:1789760747.546201:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.546214:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.546224:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.546232:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760747.546237:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.546247:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.546269:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.546278:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e97d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.546376:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cabc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515706496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36717us (37096us total) trans 4294972842 rc 0/0 00000100:00100000:1.0:1789760747.546386:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5554 00000100:00100000:1.0:1789760747.546395:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725918420436 time 725974931968 state 0x2 ref 0 00000001:00080000:3.0:1789760747.547247:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972842 is committed 00000100:00100000:2.0:1789760747.552568:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.552830:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706624 00000100:00100000:1.0:1789760747.552914:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706624 00000100:00100000:1.0:1789760747.552979:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725918420436 time 725974931968 state 0x0 ref 1 00000100:00100000:1.0:1789760747.552993:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725975192384 time 725974931968 state 0x1 ref 1 00000100:00100000:1.0:1789760747.553025:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725975192384 time 726018314329 state 0x0 ref 1 00000100:00100000:1.0:1789760747.553038:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515706624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.553087:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22634496, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760747.553096:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.553113:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.553117:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.553123:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.553137:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.580811:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972843 is committed 00002000:00010000:2.0:1789760747.580900:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22634496 -> 22638592 00010000:00010000:2.0:1789760747.580916:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.580929:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.581054:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.581064:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.581069:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.581079:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.581266:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.581276:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1e984 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.581384:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515706624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28343us (28822us total) trans 4294972843 rc 0/0 00000100:00100000:2.0:1789760747.581394:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5555 00000100:00100000:2.0:1789760747.581403:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725975192384 time 726018314329 state 0x2 ref 0 00000100:00100000:2.0:1789760747.590899:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760747.591001:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706752 00000100:00100000:0.0:1789760747.591012:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706752 00000100:00100000:0.0:1789760747.591063:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 725975192384 time 726018314329 state 0x0 ref 1 00000100:00100000:0.0:1789760747.591078:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726018574745 time 726018314329 state 0x1 ref 1 00000100:00100000:0.0:1789760747.591156:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726018574745 time 726056444400 state 0x0 ref 1 00000100:00100000:0.0:1789760747.591241:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515706752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760747.591291:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22638592, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:0.0:1789760747.591300:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.591314:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.591319:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.591325:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.591338:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.666220:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22638592 -> 22642688 00010000:00010000:0.0:1789760747.666248:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.666262:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.666271:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.666279:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760747.666444:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.666456:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.666478:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.666486:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1e98b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.666579:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515706752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75407us (75684us total) trans 4294972844 rc 0/0 00000100:00100000:0.0:1789760747.666589:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5556 00000100:00100000:0.0:1789760747.666598:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726018574745 time 726056444400 state 0x2 ref 0 00000001:00080000:1.0:1789760747.667126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972844 is committed 00000100:00100000:3.0:1789760747.669062:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760747.669208:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515706880 00000100:00100000:2.0:1789760747.669218:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515706880 00000100:00100000:2.0:1789760747.669270:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726018574745 time 726056444400 state 0x0 ref 1 00000100:00100000:2.0:1789760747.669404:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726056704816 time 726056444400 state 0x1 ref 1 00000100:00100000:2.0:1789760747.669437:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726056704816 time 726134725611 state 0x0 ref 1 00000100:00100000:2.0:1789760747.669450:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515706880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.669498:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22642688, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:2.0:1789760747.669507:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.669523:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.669527:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.669534:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.669548:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.699404:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22642688 -> 22646784 00010000:00010000:3.0:1789760747.699424:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.699444:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.699453:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.699460:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760747.699466:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.699476:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.699494:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.699502:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1e992 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.699594:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515706880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30141us (30537us total) trans 4294972845 rc 0/0 00000100:00100000:3.0:1789760747.699605:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5557 00000100:00100000:3.0:1789760747.699614:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726056704816 time 726134725611 state 0x2 ref 0 00000001:00080000:2.0:1789760747.700353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972845 is committed 00000100:00100000:0.0:1789760747.705967:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760747.707275:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707008 00000100:00100000:2.0:1789760747.707360:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707008 00000100:00100000:2.0:1789760747.707418:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726056704816 time 726134725611 state 0x0 ref 1 00000100:00100000:2.0:1789760747.707432:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726134986027 time 726134725611 state 0x1 ref 1 00000100:00100000:2.0:1789760747.707467:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726134986027 time 726172756301 state 0x0 ref 1 00000100:00100000:2.0:1789760747.707479:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.707527:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22646784, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:2.0:1789760747.707536:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707553:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.707558:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.707563:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707577:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760747.744423:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22646784 -> 22650880 00010000:00010000:2.0:1789760747.744442:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.744455:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.744464:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.744471:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.744476:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.744486:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.744504:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.744512:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e999 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.744603:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515707008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37122us (38687us total) trans 4294972846 rc 0/0 00000100:00100000:2.0:1789760747.744616:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5558 00000100:00100000:2.0:1789760747.744626:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726134986027 time 726172756301 state 0x2 ref 0 00000001:00080000:3.0:1789760747.745391:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972846 is committed 00000100:00100000:3.0:1789760747.751022:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.751491:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707136 00000100:00100000:1.0:1789760747.751501:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707136 00000100:00100000:1.0:1789760747.751555:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726134986027 time 726172756301 state 0x0 ref 1 00000100:00100000:1.0:1789760747.751571:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726173016717 time 726172756301 state 0x1 ref 1 00000100:00100000:1.0:1789760747.751784:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726173016717 time 726216996354 state 0x0 ref 1 00000100:00100000:1.0:1789760747.751799:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515707136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.751849:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22650880, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760747.751869:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.751885:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.751890:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.751896:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.751910:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.784502:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972847 is committed 00002000:00010000:3.0:1789760747.784652:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22650880 -> 22654976 00010000:00010000:3.0:1789760747.784668:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.784681:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.784690:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.784698:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760747.784703:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.784731:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.784750:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.784759:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.784958:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515707136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33156us (33939us total) trans 4294972847 rc 0/0 00000100:00100000:3.0:1789760747.784969:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5559 00000100:00100000:3.0:1789760747.784979:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726173016717 time 726216996354 state 0x2 ref 0 00000100:00100000:0.0:1789760747.789380:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.789492:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707264 00000100:00100000:1.0:1789760747.789500:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707264 00000100:00100000:1.0:1789760747.789548:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726173016717 time 726216996354 state 0x0 ref 1 00000100:00100000:1.0:1789760747.789565:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726217256770 time 726216996354 state 0x1 ref 1 00000100:00100000:1.0:1789760747.789839:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726217256770 time 726255123458 state 0x0 ref 1 00000100:00100000:1.0:1789760747.789964:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.790025:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22654976, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:1.0:1789760747.790035:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.790053:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.790058:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.790065:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.790080:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.825062:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972848 is committed 00002000:00010000:2.0:1789760747.825237:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22654976 -> 22659072 00010000:00010000:2.0:1789760747.825252:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.825264:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.825272:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.825279:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760747.825356:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.825365:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.825384:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.825391:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.825522:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35667us (36146us total) trans 4294972848 rc 0/0 00000100:00100000:2.0:1789760747.825533:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5560 00000100:00100000:2.0:1789760747.825542:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726217256770 time 726255123458 state 0x2 ref 0 00000100:00100000:3.0:1789760747.830024:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760747.830368:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707392 00000100:00100000:2.0:1789760747.830378:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707392 00000100:00100000:2.0:1789760747.830433:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726217256770 time 726255123458 state 0x0 ref 1 00000100:00100000:2.0:1789760747.830451:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726255383874 time 726255123458 state 0x1 ref 1 00000100:00100000:2.0:1789760747.830487:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726255383874 time 726295775197 state 0x0 ref 1 00000100:00100000:2.0:1789760747.830498:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.830551:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22659072, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xb007 00010000:00010000:2.0:1789760747.830559:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.830575:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.830592:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.830598:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.830612:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.838451:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760747.864455:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972849 is committed 00002000:00010000:1.0:1789760747.864561:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22659072 -> 22663168 00002000:00010000:1.0:1789760747.864566:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760747 -> 1789760748 00002000:00010000:1.0:1789760747.864569:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760747 -> 1789760748 00010000:00010000:1.0:1789760747.864653:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.864665:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.864678:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.864684:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760747.864688:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.864698:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.864781:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.864790:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1e9ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.864991:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34489us (34971us total) trans 4294972849 rc 0/0 00000100:00100000:1.0:1789760747.865001:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5561 00000100:00100000:1.0:1789760747.865011:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726255383874 time 726295775197 state 0x2 ref 0 00000100:00100000:0.0:1789760747.869413:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760747.869521:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707520 00000100:00100000:2.0:1789760747.869531:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707520 00000100:00100000:2.0:1789760747.869645:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726255383874 time 726295775197 state 0x0 ref 1 00000100:00100000:2.0:1789760747.869662:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726296035613 time 726295775197 state 0x1 ref 1 00000100:00100000:2.0:1789760747.869691:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726296035613 time 726334980193 state 0x0 ref 1 00000100:00100000:2.0:1789760747.869703:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760747.869984:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22663168, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760747.869993:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.870008:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.870012:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.870018:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.870230:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.911931:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972850 is committed 00002000:00010000:0.0:1789760747.912048:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22663168 -> 22667264 00010000:00010000:0.0:1789760747.912065:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.912076:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.912085:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.912091:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760747.912096:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.912104:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.912121:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.912129:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1e9b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.912218:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42513us (42810us total) trans 4294972850 rc 0/0 00000100:00100000:0.0:1789760747.912229:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5562 00000100:00100000:0.0:1789760747.912238:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726296035613 time 726334980193 state 0x2 ref 0 00000100:00100000:0.0:1789760747.915662:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760747.915784:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707648 00000100:00100000:3.0:1789760747.915794:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707648 00000100:00100000:3.0:1789760747.915844:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726296035613 time 726334980193 state 0x0 ref 1 00000100:00100000:3.0:1789760747.915860:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726335240609 time 726334980193 state 0x1 ref 1 00000100:00100000:3.0:1789760747.915891:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726335240609 time 726381179860 state 0x0 ref 1 00000100:00100000:3.0:1789760747.915903:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760747.915959:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22667264, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760747.915968:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.915985:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760747.915989:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.915995:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.916008:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.957412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972851 is committed 00002000:00010000:3.0:1789760747.957573:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22667264 -> 22671360 00010000:00010000:3.0:1789760747.957600:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.957619:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.957629:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.957636:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760747.957640:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.957648:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.957666:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.957673:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.958585:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42670us (42918us total) trans 4294972851 rc 0/0 00000100:00100000:3.0:1789760747.958596:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5563 00000100:00100000:3.0:1789760747.958605:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726335240609 time 726381179860 state 0x2 ref 0 00000100:00100000:3.0:1789760747.964379:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760747.964540:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707776 00000100:00100000:1.0:1789760747.964551:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707776 00000100:00100000:1.0:1789760747.964624:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726335240609 time 726381179860 state 0x0 ref 1 00000100:00100000:1.0:1789760747.964639:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726381440276 time 726381179860 state 0x1 ref 1 00000100:00100000:1.0:1789760747.964675:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726381440276 time 726429962975 state 0x0 ref 1 00000100:00100000:1.0:1789760747.964693:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760747.964779:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22671360, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:1.0:1789760747.964793:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.964809:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.964813:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.964818:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.964832:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.006325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972852 is committed 00002000:00010000:2.0:1789760748.006687:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22671360 -> 22675456 00010000:00010000:2.0:1789760748.006703:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.006741:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.006751:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.006758:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.006763:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.006772:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.006791:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.006799:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1e9c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.006991:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515707776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42300us (42616us total) trans 4294972852 rc 0/0 00000100:00100000:2.0:1789760748.007002:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5564 00000100:00100000:2.0:1789760748.007012:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726381440276 time 726429962975 state 0x2 ref 0 00000100:00100000:2.0:1789760748.010735:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.010813:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515707904 00000100:00100000:3.0:1789760748.010822:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515707904 00000100:00100000:3.0:1789760748.010898:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726381440276 time 726429962975 state 0x0 ref 1 00000100:00100000:3.0:1789760748.010912:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726430223391 time 726429962975 state 0x1 ref 1 00000100:00100000:3.0:1789760748.010953:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726430223391 time 726476241857 state 0x0 ref 1 00000100:00100000:3.0:1789760748.010966:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515707904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.011010:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22675456, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.011018:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.011034:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.011038:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.011045:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.011060:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760748.011599:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760748.082040:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972853 is committed 00002000:00010000:2.0:1789760748.082151:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22675456 -> 22679552 00010000:00010000:2.0:1789760748.082167:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.082180:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.082191:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.082208:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.082213:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.082223:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.082355:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.082364:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1e9ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.082491:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515707904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71522us (71782us total) trans 4294972853 rc 0/0 00000100:00100000:2.0:1789760748.082505:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5565 00000100:00100000:2.0:1789760748.082515:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726430223391 time 726476241857 state 0x2 ref 0 00000100:00100000:2.0:1789760748.089201:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760748.089360:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708032 00000100:00100000:1.0:1789760748.089369:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708032 00000100:00100000:1.0:1789760748.089420:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726430223391 time 726476241857 state 0x0 ref 1 00000100:00100000:1.0:1789760748.089434:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726476502273 time 726476241857 state 0x1 ref 1 00000100:00100000:1.0:1789760748.089465:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726476502273 time 726554754220 state 0x0 ref 1 00000100:00100000:1.0:1789760748.089477:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760748.089522:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22679552, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:1.0:1789760748.089530:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.089546:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760748.089550:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.089556:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.089569:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.141162:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972854 is committed 00002000:00010000:3.0:1789760748.141403:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22679552 -> 22683648 00010000:00010000:3.0:1789760748.141420:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.141439:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.141448:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.141456:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760748.141461:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.141470:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.141490:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.141498:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1e9d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.141595:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52115us (52398us total) trans 4294972854 rc 0/0 00000100:00100000:3.0:1789760748.141605:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5566 00000100:00100000:3.0:1789760748.141615:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726476502273 time 726554754220 state 0x2 ref 0 00000100:00100000:2.0:1789760748.145041:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760748.145133:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708160 00000100:00100000:1.0:1789760748.145144:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708160 00000100:00100000:1.0:1789760748.145198:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726476502273 time 726554754220 state 0x0 ref 1 00000100:00100000:1.0:1789760748.145215:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726555014636 time 726554754220 state 0x1 ref 1 00000100:00100000:1.0:1789760748.145254:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726555014636 time 726610543181 state 0x0 ref 1 00000100:00100000:1.0:1789760748.145268:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515708160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760748.145326:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22683648, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:1.0:1789760748.145334:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.145351:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760748.145355:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.145362:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.145377:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.174470:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22683648 -> 22687744 00010000:00010000:1.0:1789760748.174491:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.174504:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.174514:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.174521:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760748.174527:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.174536:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.174558:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.174566:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1e9d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.174663:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515708160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29392us (29626us total) trans 4294972855 rc 0/0 00000100:00100000:1.0:1789760748.174674:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5567 00000100:00100000:1.0:1789760748.174684:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726555014636 time 726610543181 state 0x2 ref 0 00000001:00080000:2.0:1789760748.175376:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972855 is committed 00000100:00100000:1.0:1789760748.179162:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.179307:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708288 00000100:00100000:2.0:1789760748.179318:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708288 00000100:00100000:2.0:1789760748.179379:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726555014636 time 726610543181 state 0x0 ref 1 00000100:00100000:2.0:1789760748.179394:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726610803597 time 726610543181 state 0x1 ref 1 00000100:00100000:2.0:1789760748.179427:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726610803597 time 726644716310 state 0x0 ref 1 00000100:00100000:2.0:1789760748.179439:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515708288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.179488:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22687744, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760748.179497:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.179513:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760748.179518:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.179525:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.179540:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.208139:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972856 is committed 00002000:00010000:3.0:1789760748.208263:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22687744 -> 22691840 00010000:00010000:3.0:1789760748.208348:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.208363:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.208372:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.208380:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760748.208385:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.208396:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.208416:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.208424:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1e9df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.208553:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515708288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29111us (29396us total) trans 4294972856 rc 0/0 00000100:00100000:3.0:1789760748.208564:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5568 00000100:00100000:3.0:1789760748.208574:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726610803597 time 726644716310 state 0x2 ref 0 00000100:00100000:2.0:1789760748.213147:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.213442:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708416 00000100:00100000:3.0:1789760748.213452:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708416 00000100:00100000:3.0:1789760748.213507:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726610803597 time 726644716310 state 0x0 ref 1 00000100:00100000:3.0:1789760748.213523:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726644976726 time 726644716310 state 0x1 ref 1 00000100:00100000:3.0:1789760748.213558:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726644976726 time 726678846636 state 0x0 ref 1 00000100:00100000:3.0:1789760748.213570:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.213617:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22691840, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.213626:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.213642:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.213657:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.213664:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.213676:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.244339:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972857 is committed 00002000:00010000:2.0:1789760748.244514:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22691840 -> 22695936 00010000:00010000:2.0:1789760748.244529:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.244542:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.244551:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.244559:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.244564:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.244573:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.244590:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.244598:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1e9e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.244964:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31391us (31822us total) trans 4294972857 rc 0/0 00000100:00100000:2.0:1789760748.244976:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5569 00000100:00100000:2.0:1789760748.244986:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726644976726 time 726678846636 state 0x2 ref 0 00000100:00100000:2.0:1789760748.249438:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.249660:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708544 00000100:00100000:3.0:1789760748.249671:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708544 00000100:00100000:3.0:1789760748.249902:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726644976726 time 726678846636 state 0x0 ref 1 00000100:00100000:3.0:1789760748.249917:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726679107052 time 726678846636 state 0x1 ref 1 00000100:00100000:3.0:1789760748.249995:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726679107052 time 726715283384 state 0x0 ref 1 00000100:00100000:3.0:1789760748.250010:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515708544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.250055:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22695936, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.250064:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.250081:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.250085:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.250092:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.250106:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.300046:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972858 is committed 00002000:00010000:2.0:1789760748.300159:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22695936 -> 22700032 00010000:00010000:2.0:1789760748.300174:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.300186:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.300196:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.300203:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.300208:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.300218:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.300237:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.300246:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1e9ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.300343:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515708544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50331us (50909us total) trans 4294972858 rc 0/0 00000100:00100000:2.0:1789760748.300354:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5570 00000100:00100000:2.0:1789760748.300364:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726679107052 time 726715283384 state 0x2 ref 0 00000100:00100000:0.0:1789760748.306389:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.306468:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708672 00000100:00100000:3.0:1789760748.306478:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708672 00000100:00100000:3.0:1789760748.306543:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726679107052 time 726715283384 state 0x0 ref 1 00000100:00100000:3.0:1789760748.306561:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726715543800 time 726715283384 state 0x1 ref 1 00000100:00100000:3.0:1789760748.306600:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726715543800 time 726771888783 state 0x0 ref 1 00000100:00100000:3.0:1789760748.306613:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.306661:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22700032, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.306845:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.306929:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.306943:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.306950:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.306965:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.333150:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972859 is committed 00002000:00010000:2.0:1789760748.333232:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22700032 -> 22704128 00010000:00010000:2.0:1789760748.333247:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.333258:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.333267:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.333274:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.333280:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.333289:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.333307:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.333320:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1e9f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.333445:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26830us (27060us total) trans 4294972859 rc 0/0 00000100:00100000:2.0:1789760748.333456:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5571 00000100:00100000:2.0:1789760748.333465:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726715543800 time 726771888783 state 0x2 ref 0 00000100:00100000:0.0:1789760748.339916:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.341838:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708800 00000100:00100000:2.0:1789760748.341913:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708800 00000100:00100000:2.0:1789760748.341984:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726715543800 time 726771888783 state 0x0 ref 1 00000100:00100000:2.0:1789760748.342000:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726772149199 time 726771888783 state 0x1 ref 1 00000100:00100000:2.0:1789760748.342083:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726772149199 time 726807371632 state 0x0 ref 1 00000100:00100000:2.0:1789760748.342098:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515708800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.342143:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22704128, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760748.342152:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.342166:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760748.342171:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.342178:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.342199:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.368420:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22704128 -> 22708224 00010000:00010000:2.0:1789760748.368440:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.368452:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.368463:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.368472:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.368477:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.368487:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.368507:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.368515:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1e9fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.368618:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515708800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26519us (28707us total) trans 4294972860 rc 0/0 00000100:00100000:2.0:1789760748.368629:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5572 00000100:00100000:2.0:1789760748.368640:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726772149199 time 726807371632 state 0x2 ref 0 00000001:00080000:1.0:1789760748.369169:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972860 is committed 00000100:00100000:0.0:1789760748.372441:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760748.372529:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515708928 00000100:00100000:1.0:1789760748.372538:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515708928 00000100:00100000:1.0:1789760748.372600:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726772149199 time 726807371632 state 0x0 ref 1 00000100:00100000:1.0:1789760748.372616:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726807632048 time 726807371632 state 0x1 ref 1 00000100:00100000:1.0:1789760748.372647:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726807632048 time 726837935231 state 0x0 ref 1 00000100:00100000:1.0:1789760748.372660:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515708928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760748.372746:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22708224, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:1.0:1789760748.372755:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.372772:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760748.372776:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.372783:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.372798:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.402610:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22708224 -> 22712320 00010000:00010000:3.0:1789760748.402629:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.402642:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.402652:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.402659:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760748.402673:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.402683:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.402730:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.402738:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ea02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.402829:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515708928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30167us (30393us total) trans 4294972861 rc 0/0 00000100:00100000:3.0:1789760748.402839:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5573 00000100:00100000:3.0:1789760748.402857:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726807632048 time 726837935231 state 0x2 ref 0 00000001:00080000:1.0:1789760748.403321:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972861 is committed 00000100:00100000:0.0:1789760748.407956:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.408127:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709056 00000100:00100000:3.0:1789760748.408140:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709056 00000100:00100000:3.0:1789760748.408198:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726807632048 time 726837935231 state 0x0 ref 1 00000100:00100000:3.0:1789760748.408213:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726838195647 time 726837935231 state 0x1 ref 1 00000100:00100000:3.0:1789760748.408246:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726838195647 time 726873535398 state 0x0 ref 1 00000100:00100000:3.0:1789760748.408259:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515709056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.408310:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22712320, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.408319:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.408333:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.408338:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.408344:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.408359:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.462275:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22712320 -> 22716416 00010000:00010000:3.0:1789760748.462293:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.462305:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.462314:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.462322:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760748.462327:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.462336:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.462353:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.462360:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.462448:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515709056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54186us (54496us total) trans 4294972862 rc 0/0 00000100:00100000:3.0:1789760748.462459:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5574 00000100:00100000:3.0:1789760748.462468:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726838195647 time 726873535398 state 0x2 ref 0 00000001:00080000:2.0:1789760748.466430:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972862 is committed 00000100:00100000:0.0:1789760748.467449:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.467516:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709184 00000100:00100000:3.0:1789760748.467525:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709184 00000100:00100000:3.0:1789760748.467584:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726838195647 time 726873535398 state 0x0 ref 1 00000100:00100000:3.0:1789760748.467600:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726873795814 time 726873535398 state 0x1 ref 1 00000100:00100000:3.0:1789760748.467626:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726873795814 time 726932915340 state 0x0 ref 1 00000100:00100000:3.0:1789760748.467638:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515709184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.467694:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22716416, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.467703:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467756:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.467760:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.467767:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.467781:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.501187:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972863 is committed 00002000:00010000:2.0:1789760748.501288:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22716416 -> 22720512 00010000:00010000:2.0:1789760748.501305:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.501318:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.501328:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.501335:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.501340:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.501349:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.501369:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.501382:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ea10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.501471:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515709184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33830us (34022us total) trans 4294972863 rc 0/0 00000100:00100000:2.0:1789760748.501480:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5575 00000100:00100000:2.0:1789760748.501490:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726873795814 time 726932915340 state 0x2 ref 0 00000100:00100000:2.0:1789760748.506135:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.506279:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709312 00000100:00100000:3.0:1789760748.506289:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709312 00000100:00100000:3.0:1789760748.506338:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726873795814 time 726932915340 state 0x0 ref 1 00000100:00100000:3.0:1789760748.506353:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726933175756 time 726932915340 state 0x1 ref 1 00000100:00100000:3.0:1789760748.506423:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726933175756 time 726971711811 state 0x0 ref 1 00000100:00100000:3.0:1789760748.506435:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.506481:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22720512, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.506489:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.506504:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760748.506509:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.506515:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.506528:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.579061:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972864 is committed 00002000:00010000:3.0:1789760748.579210:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22720512 -> 22724608 00010000:00010000:3.0:1789760748.579225:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.579243:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.579252:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.579260:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760748.579264:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.579273:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.579363:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.579371:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.579464:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73024us (73332us total) trans 4294972864 rc 0/0 00000100:00100000:3.0:1789760748.579474:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5576 00000100:00100000:3.0:1789760748.579484:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726933175756 time 726971711811 state 0x2 ref 0 00000100:00100000:0.0:1789760748.585574:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.585736:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709440 00000100:00100000:2.0:1789760748.585746:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709440 00000100:00100000:2.0:1789760748.585798:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726933175756 time 726971711811 state 0x0 ref 1 00000100:00100000:2.0:1789760748.585813:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726971972227 time 726971711811 state 0x1 ref 1 00000100:00100000:2.0:1789760748.585837:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726971972227 time 727051126559 state 0x0 ref 1 00000100:00100000:2.0:1789760748.585860:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.585905:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22724608, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760748.585914:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.585938:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.585943:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.585950:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.585965:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.618754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972865 is committed 00002000:00010000:1.0:1789760748.619028:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22724608 -> 22728704 00010000:00010000:1.0:1789760748.619044:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619057:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619066:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619073:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760748.619078:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619087:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619107:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.619115:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ea1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.619204:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33351us (33633us total) trans 4294972865 rc 0/0 00000100:00100000:1.0:1789760748.619215:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5577 00000100:00100000:1.0:1789760748.619224:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726971972227 time 727051126559 state 0x2 ref 0 00000100:00100000:0.0:1789760748.622667:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.622843:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709568 00000100:00100000:2.0:1789760748.622909:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709568 00000100:00100000:2.0:1789760748.622972:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 726971972227 time 727051126559 state 0x0 ref 1 00000100:00100000:2.0:1789760748.622988:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727051386975 time 727051126559 state 0x1 ref 1 00000100:00100000:2.0:1789760748.623019:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727051386975 time 727088307972 state 0x0 ref 1 00000100:00100000:2.0:1789760748.623031:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515709568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.623078:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22728704, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760748.623087:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.623103:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.623108:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.623115:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.623130:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.642833:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972866 is committed 00002000:00010000:0.0:1789760748.642943:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22728704 -> 22732800 00010000:00010000:0.0:1789760748.642959:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.642979:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.643057:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.643070:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760748.643075:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.643084:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.643105:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.643114:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ea25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.643484:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515709568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 20450us (20821us total) trans 4294972866 rc 0/0 00000100:00100000:0.0:1789760748.643495:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5578 00000100:00100000:0.0:1789760748.643505:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727051386975 time 727088307972 state 0x2 ref 0 00000100:00100000:1.0:1789760748.648243:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.648333:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709696 00000100:00100000:3.0:1789760748.648343:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709696 00000100:00100000:3.0:1789760748.648397:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727051386975 time 727088307972 state 0x0 ref 1 00000100:00100000:3.0:1789760748.648412:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727088568388 time 727088307972 state 0x1 ref 1 00000100:00100000:3.0:1789760748.648480:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727088568388 time 727113769562 state 0x0 ref 1 00000100:00100000:3.0:1789760748.648490:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.648529:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22732800, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.648535:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.648547:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.648551:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.648555:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.648565:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.694818:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972867 is committed 00002000:00010000:0.0:1789760748.694997:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22732800 -> 22736896 00010000:00010000:0.0:1789760748.695013:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.695025:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.695035:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.695043:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760748.695048:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.695058:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.695078:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.695086:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ea2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.695174:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46679us (46936us total) trans 4294972867 rc 0/0 00000100:00100000:0.0:1789760748.695184:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5579 00000100:00100000:0.0:1789760748.695194:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727088568388 time 727113769562 state 0x2 ref 0 00000100:00100000:1.0:1789760748.698504:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.698599:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709824 00000100:00100000:3.0:1789760748.698618:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709824 00000100:00100000:3.0:1789760748.698672:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727088568388 time 727113769562 state 0x0 ref 1 00000100:00100000:3.0:1789760748.698687:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727114029978 time 727113769562 state 0x1 ref 1 00000100:00100000:3.0:1789760748.698745:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727114029978 time 727164033347 state 0x0 ref 1 00000100:00100000:3.0:1789760748.698760:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.698809:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22736896, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.698818:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.698834:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.698839:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.698845:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.698858:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760748.724856:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22736896 -> 22740992 00010000:00010000:0.0:1789760748.724876:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.724888:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.724897:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.724905:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760748.724910:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.724919:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.724955:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.724963:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ea33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.725043:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972868 is committed 00000100:00100000:0.0:1789760748.725066:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515709824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26302us (26565us total) trans 4294972868 rc 0/0 00000100:00100000:0.0:1789760748.725077:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5580 00000100:00100000:0.0:1789760748.725088:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727114029978 time 727164033347 state 0x2 ref 0 00000100:00100000:2.0:1789760748.733071:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760748.733241:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515709952 00000100:00100000:3.0:1789760748.733252:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515709952 00000100:00100000:3.0:1789760748.733304:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727114029978 time 727164033347 state 0x0 ref 1 00000100:00100000:3.0:1789760748.733320:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727164293763 time 727164033347 state 0x1 ref 1 00000100:00100000:3.0:1789760748.733394:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727164293763 time 727198682398 state 0x0 ref 1 00000100:00100000:3.0:1789760748.733808:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515709952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760748.733869:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22740992, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:3.0:1789760748.733878:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.733896:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.733900:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.733906:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.733918:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.761993:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972869 is committed 00002000:00010000:2.0:1789760748.761999:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22740992 -> 22745088 00010000:00010000:2.0:1789760748.762014:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.762028:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.762037:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.762045:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.762050:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.762058:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.762076:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.762083:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ea3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.762173:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515709952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28365us (29105us total) trans 4294972869 rc 0/0 00000100:00100000:2.0:1789760748.762186:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5581 00000100:00100000:2.0:1789760748.762196:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727164293763 time 727198682398 state 0x2 ref 0 00000100:00100000:1.0:1789760748.765468:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.765570:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710080 00000100:00100000:2.0:1789760748.765579:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710080 00000100:00100000:2.0:1789760748.765630:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727164293763 time 727198682398 state 0x0 ref 1 00000100:00100000:2.0:1789760748.765646:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727198942814 time 727198682398 state 0x1 ref 1 00000100:00100000:2.0:1789760748.765676:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727198942814 time 727230965290 state 0x0 ref 1 00000100:00100000:2.0:1789760748.765688:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.765799:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22745088, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760748.765808:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.765825:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.765829:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.765835:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.765989:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.805666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972870 is committed 00002000:00010000:2.0:1789760748.806803:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22745088 -> 22749184 00010000:00010000:2.0:1789760748.806819:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.806831:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.806925:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.806941:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.806945:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.806954:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.806976:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.806983:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.807136:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41443us (41671us total) trans 4294972870 rc 0/0 00000100:00100000:2.0:1789760748.807147:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5582 00000100:00100000:2.0:1789760748.807157:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727198942814 time 727230965290 state 0x2 ref 0 00000100:00100000:1.0:1789760748.816034:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.816839:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710208 00000100:00100000:2.0:1789760748.816851:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710208 00000100:00100000:2.0:1789760748.816904:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727198942814 time 727230965290 state 0x0 ref 1 00000100:00100000:2.0:1789760748.816920:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727231225706 time 727230965290 state 0x1 ref 1 00000100:00100000:2.0:1789760748.817018:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727231225706 time 727282305890 state 0x0 ref 1 00000100:00100000:2.0:1789760748.817031:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.817079:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22749184, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xb007 00010000:00010000:2.0:1789760748.817088:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.817104:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.817109:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.817117:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.817131:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.860549:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972871 is committed 00002000:00010000:2.0:1789760748.860657:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22749184 -> 22753280 00002000:00010000:2.0:1789760748.860662:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760748 -> 1789760749 00002000:00010000:2.0:1789760748.860664:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760748 -> 1789760749 00010000:00010000:2.0:1789760748.860678:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.860691:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.860700:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.860834:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.860839:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.861257:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.861279:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.861287:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.861426:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44392us (45396us total) trans 4294972871 rc 0/0 00000100:00100000:2.0:1789760748.861438:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5583 00000100:00100000:2.0:1789760748.861447:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727231225706 time 727282305890 state 0x2 ref 0 00010000:00010000:3.0:1789760748.864536:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760748.866858:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760748.866957:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710336 00000100:00100000:2.0:1789760748.866969:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710336 00000100:00100000:2.0:1789760748.867029:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727231225706 time 727282305890 state 0x0 ref 1 00000100:00100000:2.0:1789760748.867047:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727282566306 time 727282305890 state 0x1 ref 1 00000100:00100000:2.0:1789760748.867079:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727282566306 time 727332367539 state 0x0 ref 1 00000100:00100000:2.0:1789760748.867092:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515710336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760748.867137:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22753280, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:2.0:1789760748.867156:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.867173:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.867178:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.867184:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.867198:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.902910:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22753280 -> 22757376 00010000:00010000:2.0:1789760748.902940:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.902954:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.902962:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.902969:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760748.902974:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.902983:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.903001:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.903008:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ea4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.903101:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515710336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36006us (36246us total) trans 4294972872 rc 0/0 00000100:00100000:2.0:1789760748.903112:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5584 00000100:00100000:2.0:1789760748.903123:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727282566306 time 727332367539 state 0x2 ref 0 00000001:00080000:3.0:1789760748.903437:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972872 is committed 00000100:00100000:1.0:1789760748.910669:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760748.910751:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710464 00000100:00100000:0.0:1789760748.910761:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710464 00000100:00100000:0.0:1789760748.910811:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727282566306 time 727332367539 state 0x0 ref 1 00000100:00100000:0.0:1789760748.910827:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727332627955 time 727332367539 state 0x1 ref 1 00000100:00100000:0.0:1789760748.910852:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727332627955 time 727376141684 state 0x0 ref 1 00000100:00100000:0.0:1789760748.910865:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515710464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760748.910912:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22757376, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760748.910921:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.910947:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760748.910952:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.910958:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.910973:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.947164:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22757376 -> 22761472 00010000:00010000:1.0:1789760748.947185:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.947212:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.947221:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.947228:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760748.947233:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.947248:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.947268:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.947275:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ea56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.947365:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515710464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36498us (36702us total) trans 4294972873 rc 0/0 00000100:00100000:1.0:1789760748.947375:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5585 00000100:00100000:1.0:1789760748.947384:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727332627955 time 727376141684 state 0x2 ref 0 00000001:00080000:0.0:1789760748.950396:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972873 is committed 00000100:00100000:1.0:1789760748.955438:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760748.955589:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710592 00000100:00100000:0.0:1789760748.955601:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710592 00000100:00100000:0.0:1789760748.955659:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727332627955 time 727376141684 state 0x0 ref 1 00000100:00100000:0.0:1789760748.955675:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727376402100 time 727376141684 state 0x1 ref 1 00000100:00100000:0.0:1789760748.955788:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727376402100 time 727421076205 state 0x0 ref 1 00000100:00100000:0.0:1789760748.955802:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760748.955855:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22761472, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760748.955864:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.955881:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760748.955886:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.955892:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.955908:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.998043:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972874 is committed 00002000:00010000:0.0:1789760748.998151:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22761472 -> 22765568 00010000:00010000:0.0:1789760748.998166:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.998178:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.998190:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.998197:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760748.998201:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.998209:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.998227:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.998234:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.998356:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42551us (42923us total) trans 4294972874 rc 0/0 00000100:00100000:0.0:1789760748.998366:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5586 00000100:00100000:0.0:1789760748.998376:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727376402100 time 727421076205 state 0x2 ref 0 00000100:00100000:2.0:1789760749.004298:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760749.004735:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710720 00000100:00100000:3.0:1789760749.004746:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710720 00000100:00100000:3.0:1789760749.004797:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727376402100 time 727421076205 state 0x0 ref 1 00000100:00100000:3.0:1789760749.004812:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727421336621 time 727421076205 state 0x1 ref 1 00000100:00100000:3.0:1789760749.004839:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727421336621 time 727470128249 state 0x0 ref 1 00000100:00100000:3.0:1789760749.004913:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515710720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760749.004965:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22765568, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:3.0:1789760749.004975:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.004991:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760749.004996:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.005003:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.005017:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.038294:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972875 is committed 00002000:00010000:0.0:1789760749.038424:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22765568 -> 22769664 00010000:00010000:0.0:1789760749.038437:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.038450:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.038456:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.038460:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760749.038463:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.038469:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.038484:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.038492:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ea64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.038621:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515710720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33707us (34328us total) trans 4294972875 rc 0/0 00000100:00100000:0.0:1789760749.038632:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5587 00000100:00100000:0.0:1789760749.038642:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727421336621 time 727470128249 state 0x2 ref 0 00000100:00100000:1.0:1789760749.041742:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760749.041794:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710848 00000100:00100000:2.0:1789760749.041804:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710848 00000100:00100000:2.0:1789760749.041855:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727421336621 time 727470128249 state 0x0 ref 1 00000100:00100000:2.0:1789760749.041870:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727470388665 time 727470128249 state 0x1 ref 1 00000100:00100000:2.0:1789760749.041901:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727470388665 time 727507190411 state 0x0 ref 1 00000100:00100000:2.0:1789760749.041914:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760749.042031:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22769664, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:2.0:1789760749.042041:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.042056:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760749.042061:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.042067:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.042081:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.054074:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760749.054097:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8063b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544413696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760749.054171:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760749.054188:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544413824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760749.054300:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760749.054663:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413696 00000100:00100000:2.0:1789760749.054671:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413696 00000100:00100000:2.0:1789760749.054779:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413824 00000100:00100000:2.0:1789760749.054782:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413824 00000100:00100000:3.0:1789760749.054967:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1789760749.054980:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544413696:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760749.055012:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760749.055067:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544413696:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (896us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760749.055074:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1789760749.055120:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8063b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544413696:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1789760749.056490:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760749.056500:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760749.056718:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789760749.056730:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544413824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760749.056754:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3835691008 avail 3798990848 00000100:00100000:2.0:1789760749.056826:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544413824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 100us (2527us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760749.056832:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789760749.056880:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544413824:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760749.069317:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972876 is committed 00002000:00010000:1.0:1789760749.069433:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22769664 -> 22773760 00010000:00010000:1.0:1789760749.069449:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.069462:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.069477:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.069484:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760749.069489:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.069498:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.069518:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.069526:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ea6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.069623:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515710848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27706us (27915us total) trans 4294972876 rc 0/0 00000100:00100000:1.0:1789760749.069632:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5588 00000100:00100000:1.0:1789760749.069641:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727470388665 time 727507190411 state 0x2 ref 0 00000100:00100000:2.0:1789760749.080410:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.080515:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515710976 00000100:00100000:1.0:1789760749.080526:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515710976 00000100:00100000:1.0:1789760749.080576:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727470388665 time 727507190411 state 0x0 ref 1 00000100:00100000:1.0:1789760749.080592:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727507450827 time 727507190411 state 0x1 ref 1 00000100:00100000:1.0:1789760749.080663:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727507450827 time 727545951395 state 0x0 ref 1 00000100:00100000:1.0:1789760749.080677:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515710976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.080740:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22773760, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:1.0:1789760749.080749:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.080765:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760749.080769:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.080776:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.080790:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.118087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972877 is committed 00002000:00010000:2.0:1789760749.118110:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22773760 -> 22777856 00010000:00010000:2.0:1789760749.118128:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.118139:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.118148:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.118155:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.118160:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.118169:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.118188:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.118196:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ea72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.118297:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515710976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37618us (37891us total) trans 4294972877 rc 0/0 00000100:00100000:2.0:1789760749.118311:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5589 00000100:00100000:2.0:1789760749.118321:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727507450827 time 727545951395 state 0x2 ref 0 00000100:00100000:2.0:1789760749.126127:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.126313:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711104 00000100:00100000:0.0:1789760749.126323:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711104 00000100:00100000:0.0:1789760749.126434:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727507450827 time 727545951395 state 0x0 ref 1 00000100:00100000:0.0:1789760749.126450:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727546211811 time 727545951395 state 0x1 ref 1 00000100:00100000:0.0:1789760749.126526:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727546211811 time 727591813955 state 0x0 ref 1 00000100:00100000:0.0:1789760749.126539:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515711104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.126587:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22777856, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.126595:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.126609:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760749.126613:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.126618:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.126632:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.166963:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22777856 -> 22781952 00010000:00010000:2.0:1789760749.166983:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.166997:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.167007:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.167019:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.167024:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.167033:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.167054:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.167061:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ea79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.167155:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515711104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40613us (41032us total) trans 4294972878 rc 0/0 00000100:00100000:2.0:1789760749.167165:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5590 00000100:00100000:2.0:1789760749.167175:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727546211811 time 727591813955 state 0x2 ref 0 00000001:00080000:3.0:1789760749.167284:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972878 is committed 00000100:00100000:1.0:1789760749.178297:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.178402:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711232 00000100:00100000:0.0:1789760749.178412:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711232 00000100:00100000:0.0:1789760749.178466:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727546211811 time 727591813955 state 0x0 ref 1 00000100:00100000:0.0:1789760749.178482:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727592074371 time 727591813955 state 0x1 ref 1 00000100:00100000:0.0:1789760749.178516:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727592074371 time 727643804861 state 0x0 ref 1 00000100:00100000:0.0:1789760749.178528:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515711232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.178580:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22781952, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.178589:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.178605:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760749.178609:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.178615:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.178628:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.236758:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972879 is committed 00002000:00010000:0.0:1789760749.236882:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22781952 -> 22786048 00010000:00010000:0.0:1789760749.236898:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.236912:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.236934:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.236943:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760749.236949:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.236959:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.236980:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.236988:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.237091:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515711232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58559us (58800us total) trans 4294972879 rc 0/0 00000100:00100000:0.0:1789760749.237102:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5591 00000100:00100000:0.0:1789760749.237111:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727592074371 time 727643804861 state 0x2 ref 0 00000100:00100000:1.0:1789760749.248536:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.248615:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711360 00000100:00100000:0.0:1789760749.248625:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711360 00000100:00100000:0.0:1789760749.248677:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727592074371 time 727643804861 state 0x0 ref 1 00000100:00100000:0.0:1789760749.248692:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727644065277 time 727643804861 state 0x1 ref 1 00000100:00100000:0.0:1789760749.248774:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727644065277 time 727714062538 state 0x0 ref 1 00000100:00100000:0.0:1789760749.248785:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515711360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.248833:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22786048, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.248842:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.248857:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760749.248862:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.248868:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.248881:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.299036:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972880 is committed 00002000:00010000:2.0:1789760749.299130:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22786048 -> 22790144 00010000:00010000:2.0:1789760749.299147:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.299161:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.299171:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.299178:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.299183:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.299194:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.299215:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.299227:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ea87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.299352:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515711360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50556us (50814us total) trans 4294972880 rc 0/0 00000100:00100000:2.0:1789760749.299362:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5592 00000100:00100000:2.0:1789760749.299373:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727644065277 time 727714062538 state 0x2 ref 0 00000100:00100000:3.0:1789760749.307822:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.307963:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711488 00000100:00100000:0.0:1789760749.307973:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711488 00000100:00100000:0.0:1789760749.308027:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727644065277 time 727714062538 state 0x0 ref 1 00000100:00100000:0.0:1789760749.308042:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727714322954 time 727714062538 state 0x1 ref 1 00000100:00100000:0.0:1789760749.308075:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727714322954 time 727773363755 state 0x0 ref 1 00000100:00100000:0.0:1789760749.308088:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515711488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.308139:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22790144, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.308150:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.308179:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760749.308184:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.308190:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.308205:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.341466:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972881 is committed 00002000:00010000:1.0:1789760749.341606:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22790144 -> 22794240 00010000:00010000:1.0:1789760749.341622:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.341634:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.341642:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.341648:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760749.341651:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.341657:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.341671:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.341675:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ea8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.341786:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515711488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33697us (33969us total) trans 4294972881 rc 0/0 00000100:00100000:1.0:1789760749.341796:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5593 00000100:00100000:1.0:1789760749.341805:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727714322954 time 727773363755 state 0x2 ref 0 00000100:00100000:1.0:1789760749.348578:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760749.348794:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711616 00000100:00100000:2.0:1789760749.348804:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711616 00000100:00100000:2.0:1789760749.348855:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727714322954 time 727773363755 state 0x0 ref 1 00000100:00100000:2.0:1789760749.348870:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727773624171 time 727773363755 state 0x1 ref 1 00000100:00100000:2.0:1789760749.348890:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727773624171 time 727814179752 state 0x0 ref 1 00000100:00100000:2.0:1789760749.348901:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515711616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760749.349127:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22794240, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:2.0:1789760749.349136:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.349150:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760749.349155:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.349161:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.349176:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.380870:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972882 is committed 00002000:00010000:2.0:1789760749.380982:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22794240 -> 22798336 00010000:00010000:2.0:1789760749.380997:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.381009:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.381018:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.381025:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.381030:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.381039:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.381058:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.381067:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ea95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.381169:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515711616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32265us (32596us total) trans 4294972882 rc 0/0 00000100:00100000:2.0:1789760749.381179:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5594 00000100:00100000:2.0:1789760749.381188:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727773624171 time 727814179752 state 0x2 ref 0 00000100:00100000:2.0:1789760749.384945:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.385082:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711744 00000100:00100000:1.0:1789760749.385092:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711744 00000100:00100000:1.0:1789760749.385149:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727773624171 time 727814179752 state 0x0 ref 1 00000100:00100000:1.0:1789760749.385163:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727814440168 time 727814179752 state 0x1 ref 1 00000100:00100000:1.0:1789760749.385193:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727814440168 time 727850482148 state 0x0 ref 1 00000100:00100000:1.0:1789760749.385205:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515711744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.385261:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22798336, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:1.0:1789760749.385270:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.385288:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760749.385293:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.385298:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.385312:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.416673:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972883 is committed 00002000:00010000:2.0:1789760749.416882:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22798336 -> 22802432 00010000:00010000:2.0:1789760749.416898:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.416910:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.416918:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.416935:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.416939:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.416948:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.416966:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.416980:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ea9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.417162:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515711744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31954us (32232us total) trans 4294972883 rc 0/0 00000100:00100000:2.0:1789760749.417172:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5595 00000100:00100000:2.0:1789760749.417181:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727814440168 time 727850482148 state 0x2 ref 0 00000100:00100000:0.0:1789760749.420810:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760749.420968:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515711872 00000100:00100000:2.0:1789760749.420979:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515711872 00000100:00100000:2.0:1789760749.421030:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727814440168 time 727850482148 state 0x0 ref 1 00000100:00100000:2.0:1789760749.421045:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727850742564 time 727850482148 state 0x1 ref 1 00000100:00100000:2.0:1789760749.421076:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727850742564 time 727886365017 state 0x0 ref 1 00000100:00100000:2.0:1789760749.421096:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515711872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760749.421144:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22802432, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:2.0:1789760749.421153:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.421168:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760749.421173:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.421180:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.421199:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.492188:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22802432 -> 22806528 00010000:00010000:3.0:1789760749.492210:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.492224:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.492235:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.492248:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760749.492253:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.492263:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.492283:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.492291:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eaa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.492384:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515711872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71286us (71579us total) trans 4294972884 rc 0/0 00000100:00100000:3.0:1789760749.492395:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5596 00000100:00100000:3.0:1789760749.492405:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727850742564 time 727886365017 state 0x2 ref 0 00000001:00080000:1.0:1789760749.492477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972884 is committed 00000100:00100000:1.0:1789760749.501544:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.501673:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712000 00000100:00100000:0.0:1789760749.501682:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712000 00000100:00100000:0.0:1789760749.501760:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727850742564 time 727886365017 state 0x0 ref 1 00000100:00100000:0.0:1789760749.501775:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727886625433 time 727886365017 state 0x1 ref 1 00000100:00100000:0.0:1789760749.501805:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727886625433 time 727967094038 state 0x0 ref 1 00000100:00100000:0.0:1789760749.501817:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515712000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.501872:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22806528, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.501881:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.501898:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760749.501902:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.501909:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.501932:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.523624:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972885 is committed 00002000:00010000:2.0:1789760749.523969:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22806528 -> 22810624 00010000:00010000:2.0:1789760749.523987:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.524000:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.524010:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.524017:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.524022:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.524030:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.524050:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.524058:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eaaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.524161:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515712000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22333us (22613us total) trans 4294972885 rc 0/0 00000100:00100000:2.0:1789760749.524171:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5597 00000100:00100000:2.0:1789760749.524180:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727886625433 time 727967094038 state 0x2 ref 0 00000100:00100000:2.0:1789760749.529185:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.532942:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712128 00000100:00100000:0.0:1789760749.532957:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712128 00000100:00100000:0.0:1789760749.533085:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727886625433 time 727967094038 state 0x0 ref 1 00000100:00100000:0.0:1789760749.533100:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727967354454 time 727967094038 state 0x1 ref 1 00000100:00100000:0.0:1789760749.533173:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727967354454 time 727998461474 state 0x0 ref 1 00000100:00100000:0.0:1789760749.533187:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515712128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.533233:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22810624, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.533241:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.533257:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760749.533261:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.533267:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.533281:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.583817:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972886 is committed 00002000:00010000:1.0:1789760749.584044:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22810624 -> 22814720 00010000:00010000:1.0:1789760749.584061:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.584075:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.584085:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.584091:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760749.584097:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.584105:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.584128:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.584136:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eab1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.584281:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515712128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51092us (55101us total) trans 4294972886 rc 0/0 00000100:00100000:1.0:1789760749.584292:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5598 00000100:00100000:1.0:1789760749.584301:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727967354454 time 727998461474 state 0x2 ref 0 00000100:00100000:0.0:1789760749.589009:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.589070:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712256 00000100:00100000:1.0:1789760749.589080:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712256 00000100:00100000:1.0:1789760749.589134:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727967354454 time 727998461474 state 0x0 ref 1 00000100:00100000:1.0:1789760749.589155:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727998721890 time 727998461474 state 0x1 ref 1 00000100:00100000:1.0:1789760749.589185:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727998721890 time 728054474280 state 0x0 ref 1 00000100:00100000:1.0:1789760749.589196:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515712256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.589243:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22814720, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:1.0:1789760749.589252:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.589268:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.589272:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.589278:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.589291:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.632996:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972887 is committed 00002000:00010000:0.0:1789760749.633142:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22814720 -> 22818816 00010000:00010000:0.0:1789760749.633158:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.633171:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.633181:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.633188:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760749.633193:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.633202:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.633221:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.633228:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eab8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.633355:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515712256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44149us (44348us total) trans 4294972887 rc 0/0 00000100:00100000:0.0:1789760749.633366:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5599 00000100:00100000:0.0:1789760749.633376:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727998721890 time 728054474280 state 0x2 ref 0 00000100:00100000:2.0:1789760749.639896:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.640019:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712384 00000100:00100000:1.0:1789760749.640030:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712384 00000100:00100000:1.0:1789760749.640130:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 727998721890 time 728054474280 state 0x0 ref 1 00000100:00100000:1.0:1789760749.640145:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728054734696 time 728054474280 state 0x1 ref 1 00000100:00100000:1.0:1789760749.640207:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728054734696 time 728105496001 state 0x0 ref 1 00000100:00100000:1.0:1789760749.640220:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515712384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.640269:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22818816, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:1.0:1789760749.640278:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.640305:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.640310:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.640317:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.640330:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.676163:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972888 is committed 00002000:00010000:1.0:1789760749.676325:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22818816 -> 22822912 00010000:00010000:1.0:1789760749.676340:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.676352:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.676361:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.676369:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760749.676374:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.676383:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.676402:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.676409:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eabf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.676503:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515712384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36281us (36612us total) trans 4294972888 rc 0/0 00000100:00100000:1.0:1789760749.676512:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5600 00000100:00100000:1.0:1789760749.676522:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728054734696 time 728105496001 state 0x2 ref 0 00000100:00100000:2.0:1789760749.680254:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.680336:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712512 00000100:00100000:1.0:1789760749.680346:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712512 00000100:00100000:1.0:1789760749.680399:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728054734696 time 728105496001 state 0x0 ref 1 00000100:00100000:1.0:1789760749.680414:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728105756417 time 728105496001 state 0x1 ref 1 00000100:00100000:1.0:1789760749.680503:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728105756417 time 728145791952 state 0x0 ref 1 00000100:00100000:1.0:1789760749.680516:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515712512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.680564:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22822912, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:1.0:1789760749.680573:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.680588:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.680593:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.680599:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.680613:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.713428:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972889 is committed 00002000:00010000:1.0:1789760749.713589:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22822912 -> 22827008 00010000:00010000:1.0:1789760749.713604:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.713617:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.713626:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.713633:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760749.713639:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.713648:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.713668:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.713676:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1eac6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.713815:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515712512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33295us (33564us total) trans 4294972889 rc 0/0 00000100:00100000:1.0:1789760749.713828:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5601 00000100:00100000:1.0:1789760749.713838:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728105756417 time 728145791952 state 0x2 ref 0 00000100:00100000:0.0:1789760749.718040:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760749.718319:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712640 00000100:00100000:3.0:1789760749.718329:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712640 00000100:00100000:3.0:1789760749.718385:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728105756417 time 728145791952 state 0x0 ref 1 00000100:00100000:3.0:1789760749.718452:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728146052368 time 728145791952 state 0x1 ref 1 00000100:00100000:3.0:1789760749.718484:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728146052368 time 728183773061 state 0x0 ref 1 00000100:00100000:3.0:1789760749.718496:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515712640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760749.718541:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22827008, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:3.0:1789760749.718550:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.718574:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760749.718579:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.718584:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.718599:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.759443:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972890 is committed 00002000:00010000:3.0:1789760749.759652:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22827008 -> 22831104 00010000:00010000:3.0:1789760749.759667:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.759680:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.759689:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.759697:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760749.759703:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.759742:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.759762:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.759770:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eacd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.759900:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515712640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41401us (41875us total) trans 4294972890 rc 0/0 00000100:00100000:3.0:1789760749.759910:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5602 00000100:00100000:3.0:1789760749.759935:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728146052368 time 728183773061 state 0x2 ref 0 00000100:00100000:3.0:1789760749.763897:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760749.764048:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712768 00000100:00100000:2.0:1789760749.764059:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712768 00000100:00100000:2.0:1789760749.764117:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728146052368 time 728183773061 state 0x0 ref 1 00000100:00100000:2.0:1789760749.764132:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728184033477 time 728183773061 state 0x1 ref 1 00000100:00100000:2.0:1789760749.764159:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728184033477 time 728229449117 state 0x0 ref 1 00000100:00100000:2.0:1789760749.764170:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515712768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760749.764216:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22831104, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:2.0:1789760749.764225:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.764240:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.764245:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.764251:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.764265:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.807347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972891 is committed 00002000:00010000:3.0:1789760749.807479:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22831104 -> 22835200 00010000:00010000:3.0:1789760749.807494:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.807508:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.807516:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.807529:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760749.807535:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.807543:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.807571:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.807579:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ead4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.807770:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515712768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43592us (43873us total) trans 4294972891 rc 0/0 00000100:00100000:3.0:1789760749.807783:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5603 00000100:00100000:3.0:1789760749.807795:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728184033477 time 728229449117 state 0x2 ref 0 00000100:00100000:2.0:1789760749.817314:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760749.817616:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515712896 00000100:00100000:0.0:1789760749.817626:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515712896 00000100:00100000:0.0:1789760749.817680:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728184033477 time 728229449117 state 0x0 ref 1 00000100:00100000:0.0:1789760749.817694:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728229709533 time 728229449117 state 0x1 ref 1 00000100:00100000:0.0:1789760749.817760:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728229709533 time 728283048639 state 0x0 ref 1 00000100:00100000:0.0:1789760749.817773:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515712896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760749.817820:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22835200, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xb007 00010000:00010000:0.0:1789760749.817828:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.817842:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760749.817846:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.817853:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.817866:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.886151:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760749.888454:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972892 is committed 00002000:00010000:3.0:1789760749.888555:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22835200 -> 22839296 00002000:00010000:3.0:1789760749.888560:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760749 -> 1789760750 00002000:00010000:3.0:1789760749.888563:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760749 -> 1789760750 00010000:00010000:3.0:1789760749.888576:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.888589:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.888598:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.888605:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760749.888610:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.888620:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.888640:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.888648:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eadb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.888762:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515712896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70987us (71455us total) trans 4294972892 rc 0/0 00000100:00100000:3.0:1789760749.888772:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5604 00000100:00100000:3.0:1789760749.888780:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728229709533 time 728283048639 state 0x2 ref 0 00000100:00100000:0.0:1789760749.892410:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760749.892778:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713024 00000100:00100000:2.0:1789760749.892787:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713024 00000100:00100000:2.0:1789760749.892849:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728229709533 time 728283048639 state 0x0 ref 1 00000100:00100000:2.0:1789760749.892866:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728283309055 time 728283048639 state 0x1 ref 1 00000100:00100000:2.0:1789760749.892896:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728283309055 time 728358184641 state 0x0 ref 1 00000100:00100000:2.0:1789760749.892908:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760749.892966:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22839296, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:2.0:1789760749.892975:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.892990:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.892994:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.893000:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.893015:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.949391:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22839296 -> 22843392 00010000:00010000:2.0:1789760749.949412:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.949426:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.949436:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.949443:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.949447:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.949455:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.949476:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.949483:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eae2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.949576:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515713024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56666us (57171us total) trans 4294972893 rc 0/0 00000100:00100000:2.0:1789760749.949591:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5605 00000100:00100000:2.0:1789760749.949600:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728283309055 time 728358184641 state 0x2 ref 0 00000001:00080000:0.0:1789760749.949698:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972893 is committed 00000100:00100000:3.0:1789760749.955832:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.955992:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713152 00000100:00100000:1.0:1789760749.956004:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713152 00000100:00100000:1.0:1789760749.956056:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728283309055 time 728358184641 state 0x0 ref 1 00000100:00100000:1.0:1789760749.956070:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728358445057 time 728358184641 state 0x1 ref 1 00000100:00100000:1.0:1789760749.956100:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728358445057 time 728421389233 state 0x0 ref 1 00000100:00100000:1.0:1789760749.956113:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.956159:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22843392, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:1.0:1789760749.956168:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.956184:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.956188:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.956194:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.956208:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.986441:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22843392 -> 22847488 00010000:00010000:2.0:1789760749.986461:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.986474:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.986496:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.986504:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760749.986510:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.986519:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.986538:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.986545:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1eae9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.986646:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515713152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30530us (30819us total) trans 4294972894 rc 0/0 00000100:00100000:2.0:1789760749.986657:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5606 00000100:00100000:2.0:1789760749.986666:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728358445057 time 728421389233 state 0x2 ref 0 00000001:00080000:0.0:1789760749.987245:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972894 is committed 00000100:00100000:3.0:1789760749.991445:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760749.991852:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713280 00000100:00100000:1.0:1789760749.991862:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713280 00000100:00100000:1.0:1789760749.991914:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728358445057 time 728421389233 state 0x0 ref 1 00000100:00100000:1.0:1789760749.991939:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728421649649 time 728421389233 state 0x1 ref 1 00000100:00100000:1.0:1789760749.991968:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728421649649 time 728457256957 state 0x0 ref 1 00000100:00100000:1.0:1789760749.991980:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760749.992025:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22847488, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:1.0:1789760749.992034:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.992049:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760749.992054:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.992059:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.992073:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.052485:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972895 is committed 00002000:00010000:3.0:1789760750.054048:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22847488 -> 22851584 00010000:00010000:3.0:1789760750.054068:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.054082:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.054091:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.054099:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.054104:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.054114:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.054133:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.054142:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1eaf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.054238:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62254us (62794us total) trans 4294972895 rc 0/0 00000100:00100000:3.0:1789760750.054248:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5607 00000100:00100000:3.0:1789760750.054257:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728421649649 time 728457256957 state 0x2 ref 0 00000100:00100000:0.0:1789760750.058087:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.058181:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713408 00000100:00100000:3.0:1789760750.058191:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713408 00000100:00100000:3.0:1789760750.058286:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728421649649 time 728457256957 state 0x0 ref 1 00000100:00100000:3.0:1789760750.058301:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728457517373 time 728457256957 state 0x1 ref 1 00000100:00100000:3.0:1789760750.058334:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728457517373 time 728523623250 state 0x0 ref 1 00000100:00100000:3.0:1789760750.058395:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.058507:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22851584, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.058517:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.058532:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760750.058536:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.058542:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.058556:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760750.059314:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760750.088660:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22851584 -> 22855680 00010000:00010000:3.0:1789760750.088682:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.088696:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.088733:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.088741:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.088758:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.088769:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.088788:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.088796:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eaf7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.088891:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515713408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30542us (30811us total) trans 4294972896 rc 0/0 00000100:00100000:3.0:1789760750.088902:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5608 00000100:00100000:3.0:1789760750.088911:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728457517373 time 728523623250 state 0x2 ref 0 00000001:00080000:1.0:1789760750.089556:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972896 is committed 00000100:00100000:0.0:1789760750.093962:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.094038:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713536 00000100:00100000:3.0:1789760750.094049:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713536 00000100:00100000:3.0:1789760750.094101:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728457517373 time 728523623250 state 0x0 ref 1 00000100:00100000:3.0:1789760750.094117:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728523883666 time 728523623250 state 0x1 ref 1 00000100:00100000:3.0:1789760750.094149:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728523883666 time 728559437960 state 0x0 ref 1 00000100:00100000:3.0:1789760750.094161:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.094208:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22855680, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.094216:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.094232:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760750.094237:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.094244:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.094258:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760750.131028:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22855680 -> 22859776 00010000:00010000:1.0:1789760750.131048:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.131062:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.131068:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.131073:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760750.131076:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.131082:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.131098:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.131103:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1eafe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.131260:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515713536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37028us (37234us total) trans 4294972897 rc 0/0 00000100:00100000:1.0:1789760750.131271:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5609 00000100:00100000:1.0:1789760750.131282:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728523883666 time 728559437960 state 0x2 ref 0 00000001:00080000:1.0:1789760750.131330:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972897 is committed 00000100:00100000:2.0:1789760750.137081:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.137225:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713664 00000100:00100000:3.0:1789760750.137235:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713664 00000100:00100000:3.0:1789760750.137288:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728523883666 time 728559437960 state 0x0 ref 1 00000100:00100000:3.0:1789760750.137304:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728559698376 time 728559437960 state 0x1 ref 1 00000100:00100000:3.0:1789760750.137336:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728559698376 time 728602624808 state 0x0 ref 1 00000100:00100000:3.0:1789760750.137348:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515713664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.137397:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22859776, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.137406:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.137421:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760750.137425:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.137548:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.137565:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.167666:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22859776 -> 22863872 00010000:00010000:0.0:1789760750.167689:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.167702:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.167728:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.167735:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760750.167739:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.167756:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.167774:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.167782:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.167880:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515713664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30529us (30802us total) trans 4294972898 rc 0/0 00000100:00100000:0.0:1789760750.167895:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5610 00000100:00100000:0.0:1789760750.167904:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728559698376 time 728602624808 state 0x2 ref 0 00000001:00080000:2.0:1789760750.168522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972898 is committed 00000100:00100000:0.0:1789760750.173116:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760750.173262:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713792 00000100:00100000:2.0:1789760750.173272:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713792 00000100:00100000:2.0:1789760750.173320:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728559698376 time 728602624808 state 0x0 ref 1 00000100:00100000:2.0:1789760750.173334:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728602885224 time 728602624808 state 0x1 ref 1 00000100:00100000:2.0:1789760750.173362:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728602885224 time 728638651363 state 0x0 ref 1 00000100:00100000:2.0:1789760750.173374:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760750.173421:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22863872, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:2.0:1789760750.173429:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.173513:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760750.173518:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.173525:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.173544:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.200676:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972899 is committed 00002000:00010000:2.0:1789760750.200912:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22863872 -> 22867968 00010000:00010000:2.0:1789760750.200938:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.200952:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.200962:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.200969:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760750.200975:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.200984:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.201006:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.201013:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1eb0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.201151:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515713792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27773us (28039us total) trans 4294972899 rc 0/0 00000100:00100000:2.0:1789760750.201163:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5611 00000100:00100000:2.0:1789760750.201173:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728602885224 time 728638651363 state 0x2 ref 0 00000100:00100000:3.0:1789760750.205516:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760750.205600:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515713920 00000100:00100000:2.0:1789760750.205609:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515713920 00000100:00100000:2.0:1789760750.205670:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728602885224 time 728638651363 state 0x0 ref 1 00000100:00100000:2.0:1789760750.205685:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728638911779 time 728638651363 state 0x1 ref 1 00000100:00100000:2.0:1789760750.205743:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728638911779 time 728671032501 state 0x0 ref 1 00000100:00100000:2.0:1789760750.205765:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515713920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760750.205813:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22867968, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:2.0:1789760750.205822:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.205837:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760750.205852:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.205858:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.205871:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.264109:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972900 is committed 00002000:00010000:2.0:1789760750.264235:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22867968 -> 22872064 00010000:00010000:2.0:1789760750.264363:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.264376:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.264385:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.264393:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760750.264398:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.264408:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.264427:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.264435:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1eb13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.264533:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515713920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58765us (59097us total) trans 4294972900 rc 0/0 00000100:00100000:2.0:1789760750.264547:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5612 00000100:00100000:2.0:1789760750.264557:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728638911779 time 728671032501 state 0x2 ref 0 00000100:00100000:2.0:1789760750.271163:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760750.271277:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714048 00000100:00100000:1.0:1789760750.271290:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714048 00000100:00100000:1.0:1789760750.271352:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728638911779 time 728671032501 state 0x0 ref 1 00000100:00100000:1.0:1789760750.271368:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728671292917 time 728671032501 state 0x1 ref 1 00000100:00100000:1.0:1789760750.271435:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728671292917 time 728736723676 state 0x0 ref 1 00000100:00100000:1.0:1789760750.271448:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760750.271507:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22872064, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:1.0:1789760750.271517:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.271534:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760750.271539:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.271546:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.271570:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.308514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972901 is committed 00002000:00010000:3.0:1789760750.308680:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22872064 -> 22876160 00010000:00010000:3.0:1789760750.308694:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.309468:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.309480:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.309487:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.309491:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.309500:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.309520:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.309527:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1eb1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.309758:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38307us (38604us total) trans 4294972901 rc 0/0 00000100:00100000:3.0:1789760750.309769:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5613 00000100:00100000:3.0:1789760750.309779:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728671292917 time 728736723676 state 0x2 ref 0 00000100:00100000:1.0:1789760750.313661:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.314287:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714176 00000100:00100000:3.0:1789760750.314296:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714176 00000100:00100000:3.0:1789760750.314343:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728671292917 time 728736723676 state 0x0 ref 1 00000100:00100000:3.0:1789760750.314357:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728736984092 time 728736723676 state 0x1 ref 1 00000100:00100000:3.0:1789760750.314429:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728736984092 time 728779716296 state 0x0 ref 1 00000100:00100000:3.0:1789760750.314441:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.314488:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22876160, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.314497:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.314513:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760750.314519:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.314526:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.314539:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.356531:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972902 is committed 00002000:00010000:3.0:1789760750.356683:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22876160 -> 22880256 00010000:00010000:3.0:1789760750.356698:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.356776:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.356785:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.356792:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.356797:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.356805:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.356824:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.356898:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.357072:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42627us (43411us total) trans 4294972902 rc 0/0 00000100:00100000:2.0:1789760750.357082:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5614 00000100:00100000:2.0:1789760750.357157:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728736984092 time 728779716296 state 0x2 ref 0 00000100:00100000:2.0:1789760750.360287:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760750.360353:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714304 00000100:00100000:1.0:1789760750.360363:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714304 00000100:00100000:1.0:1789760750.360416:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728736984092 time 728779716296 state 0x0 ref 1 00000100:00100000:1.0:1789760750.360431:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728779976712 time 728779716296 state 0x1 ref 1 00000100:00100000:1.0:1789760750.360465:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728779976712 time 728825754269 state 0x0 ref 1 00000100:00100000:1.0:1789760750.360478:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515714304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760750.360523:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22880256, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:1.0:1789760750.360532:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.360548:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760750.360553:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.360570:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.360585:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.404207:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972903 is committed 00002000:00010000:3.0:1789760750.404343:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22880256 -> 22884352 00010000:00010000:3.0:1789760750.404358:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.404372:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.404383:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.404391:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.404396:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.404406:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.404427:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.404436:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1eb28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.404583:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515714304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44102us (44299us total) trans 4294972903 rc 0/0 00000100:00100000:3.0:1789760750.404594:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5615 00000100:00100000:3.0:1789760750.404605:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728779976712 time 728825754269 state 0x2 ref 0 00000100:00100000:0.0:1789760750.408237:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.408327:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714432 00000100:00100000:3.0:1789760750.408337:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714432 00000100:00100000:3.0:1789760750.408392:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728779976712 time 728825754269 state 0x0 ref 1 00000100:00100000:3.0:1789760750.408408:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728826014685 time 728825754269 state 0x1 ref 1 00000100:00100000:3.0:1789760750.408481:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728826014685 time 728873769108 state 0x0 ref 1 00000100:00100000:3.0:1789760750.408494:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.408543:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22884352, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.408553:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.408579:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760750.408583:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.408591:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.408604:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.441796:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22884352 -> 22888448 00010000:00010000:3.0:1789760750.441822:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.441847:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.441857:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.441865:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.441871:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.441881:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.441902:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.441910:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.442029:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515714432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33532us (33798us total) trans 4294972904 rc 0/0 00000100:00100000:3.0:1789760750.442044:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5616 00000100:00100000:3.0:1789760750.442053:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728826014685 time 728873769108 state 0x2 ref 0 00000100:00100000:1.0:1789760750.446540:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760750.446750:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714560 00000100:00100000:0.0:1789760750.446761:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714560 00000100:00100000:0.0:1789760750.446817:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728826014685 time 728873769108 state 0x0 ref 1 00000100:00100000:0.0:1789760750.446843:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728874029524 time 728873769108 state 0x1 ref 1 00000100:00100000:0.0:1789760750.446870:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728874029524 time 728912158954 state 0x0 ref 1 00000100:00100000:0.0:1789760750.446882:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515714560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760750.446939:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22888448, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:0.0:1789760750.446948:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.446964:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760750.446969:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.446975:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.446993:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.448253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972904 is committed 00000001:00080000:3.0:1789760750.476203:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972905 is committed 00002000:00010000:2.0:1789760750.476302:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22888448 -> 22892544 00010000:00010000:2.0:1789760750.476318:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.476331:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.476340:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.476347:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760750.476351:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.476360:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.476376:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.476385:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eb36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.476472:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29587us (29938us total) trans 4294972905 rc 0/0 00000100:00100000:2.0:1789760750.476481:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5617 00000100:00100000:2.0:1789760750.476491:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728874029524 time 728912158954 state 0x2 ref 0 00000100:00100000:1.0:1789760750.479903:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.480040:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714688 00000100:00100000:3.0:1789760750.480050:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714688 00000100:00100000:3.0:1789760750.480104:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728874029524 time 728912158954 state 0x0 ref 1 00000100:00100000:3.0:1789760750.480119:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728912419370 time 728912158954 state 0x1 ref 1 00000100:00100000:3.0:1789760750.480152:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728912419370 time 728945440339 state 0x0 ref 1 00000100:00100000:3.0:1789760750.480164:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515714688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.480213:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22892544, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.480222:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.480238:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760750.480242:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.480249:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.480262:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760750.508563:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22892544 -> 22896640 00010000:00010000:1.0:1789760750.508582:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.508596:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.508606:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.508619:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760750.508624:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.508633:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.508650:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.508658:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1eb3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.508819:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515714688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28653us (28921us total) trans 4294972906 rc 0/0 00000100:00100000:1.0:1789760750.508829:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5618 00000100:00100000:1.0:1789760750.508838:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728912419370 time 728945440339 state 0x2 ref 0 00000001:00080000:3.0:1789760750.509111:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972906 is committed 00000100:00100000:3.0:1789760750.514027:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760750.514458:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714816 00000100:00100000:1.0:1789760750.514469:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714816 00000100:00100000:1.0:1789760750.514522:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728912419370 time 728945440339 state 0x0 ref 1 00000100:00100000:1.0:1789760750.514538:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728945700755 time 728945440339 state 0x1 ref 1 00000100:00100000:1.0:1789760750.514567:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728945700755 time 728979856295 state 0x0 ref 1 00000100:00100000:1.0:1789760750.514582:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760750.514631:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22896640, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:1.0:1789760750.514640:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.515433:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760750.515439:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.515446:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.515460:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.550302:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972907 is committed 00002000:00010000:1.0:1789760750.550505:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22896640 -> 22900736 00010000:00010000:1.0:1789760750.550523:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.550536:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.550546:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.550553:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760750.550558:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.550567:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.550590:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.550598:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.550980:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515714816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36395us (36958us total) trans 4294972907 rc 0/0 00000100:00100000:1.0:1789760750.550991:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5619 00000100:00100000:1.0:1789760750.551001:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728945700755 time 728979856295 state 0x2 ref 0 00000100:00100000:1.0:1789760750.554678:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.554775:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515714944 00000100:00100000:3.0:1789760750.554786:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515714944 00000100:00100000:3.0:1789760750.554836:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728945700755 time 728979856295 state 0x0 ref 1 00000100:00100000:3.0:1789760750.554851:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728980116711 time 728979856295 state 0x1 ref 1 00000100:00100000:3.0:1789760750.554882:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728980116711 time 729020172029 state 0x0 ref 1 00000100:00100000:3.0:1789760750.554894:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515714944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.554949:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22900736, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.554958:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.554974:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.554979:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.554986:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.555000:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.587977:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972908 is committed 00002000:00010000:3.0:1789760750.588046:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22900736 -> 22904832 00010000:00010000:3.0:1789760750.588063:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.588080:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.588089:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.588096:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.588100:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.588109:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.588127:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.588135:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.588214:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515714944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33318us (33540us total) trans 4294972908 rc 0/0 00000100:00100000:3.0:1789760750.588223:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5620 00000100:00100000:3.0:1789760750.588233:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728980116711 time 729020172029 state 0x2 ref 0 00000100:00100000:2.0:1789760750.602214:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760750.602386:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715072 00000100:00100000:1.0:1789760750.602396:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715072 00000100:00100000:1.0:1789760750.602448:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 728980116711 time 729020172029 state 0x0 ref 1 00000100:00100000:1.0:1789760750.602463:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729020432445 time 729020172029 state 0x1 ref 1 00000100:00100000:1.0:1789760750.602494:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729020432445 time 729067783600 state 0x0 ref 1 00000100:00100000:1.0:1789760750.602506:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515715072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760750.602554:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22904832, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:1.0:1789760750.602564:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.602580:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760750.602584:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.602590:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.602604:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.635801:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972909 is committed 00002000:00010000:1.0:1789760750.636057:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22904832 -> 22908928 00010000:00010000:1.0:1789760750.636073:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.636085:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.636093:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.636100:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760750.636104:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.636112:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.636136:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.636143:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.636263:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515715072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33754us (34054us total) trans 4294972909 rc 0/0 00000100:00100000:1.0:1789760750.636273:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5621 00000100:00100000:1.0:1789760750.636282:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729020432445 time 729067783600 state 0x2 ref 0 00000100:00100000:3.0:1789760750.645660:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760750.645832:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715200 00000100:00100000:0.0:1789760750.645842:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715200 00000100:00100000:0.0:1789760750.645897:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729020432445 time 729067783600 state 0x0 ref 1 00000100:00100000:0.0:1789760750.645912:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729068044016 time 729067783600 state 0x1 ref 1 00000100:00100000:0.0:1789760750.645958:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729068044016 time 729111241008 state 0x0 ref 1 00000100:00100000:0.0:1789760750.645971:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515715200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760750.646017:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22908928, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:0.0:1789760750.646027:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.646044:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.646048:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.646054:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.646069:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.676475:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22908928 -> 22913024 00010000:00010000:3.0:1789760750.676496:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.676509:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.676518:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.676526:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.676530:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.676538:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.676557:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.676565:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1eb59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.676658:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515715200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30683us (31001us total) trans 4294972910 rc 0/0 00000100:00100000:3.0:1789760750.676669:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5622 00000100:00100000:3.0:1789760750.676678:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729068044016 time 729111241008 state 0x2 ref 0 00000001:00080000:0.0:1789760750.676795:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972910 is committed 00000100:00100000:2.0:1789760750.684966:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760750.685062:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715328 00000100:00100000:0.0:1789760750.685071:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715328 00000100:00100000:0.0:1789760750.685123:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729068044016 time 729111241008 state 0x0 ref 1 00000100:00100000:0.0:1789760750.685139:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729111501424 time 729111241008 state 0x1 ref 1 00000100:00100000:0.0:1789760750.685167:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729111501424 time 729150455808 state 0x0 ref 1 00000100:00100000:0.0:1789760750.685179:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515715328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760750.685225:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22913024, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:0.0:1789760750.685235:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.685250:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.685255:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.685262:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.685278:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.733138:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972911 is committed 00002000:00010000:0.0:1789760750.733206:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22913024 -> 22917120 00010000:00010000:0.0:1789760750.733222:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.733236:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.733245:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.733252:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760750.733257:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.733265:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.733284:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.733292:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.733376:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515715328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48195us (48423us total) trans 4294972911 rc 0/0 00000100:00100000:0.0:1789760750.733386:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5623 00000100:00100000:0.0:1789760750.733396:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729111501424 time 729150455808 state 0x2 ref 0 00000100:00100000:2.0:1789760750.737666:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.737780:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715456 00000100:00100000:3.0:1789760750.737790:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715456 00000100:00100000:3.0:1789760750.737908:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729111501424 time 729150455808 state 0x0 ref 1 00000100:00100000:3.0:1789760750.737934:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729150716224 time 729150455808 state 0x1 ref 1 00000100:00100000:3.0:1789760750.737963:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729150716224 time 729203252030 state 0x0 ref 1 00000100:00100000:3.0:1789760750.737976:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515715456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.738025:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22917120, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.738034:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.738051:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.738055:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.738060:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.738073:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.785047:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972912 is committed 00002000:00010000:3.0:1789760750.785223:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22917120 -> 22921216 00010000:00010000:3.0:1789760750.785239:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785252:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785263:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785270:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760750.785275:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785284:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785303:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785311:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.785402:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515715456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47424us (47741us total) trans 4294972912 rc 0/0 00000100:00100000:3.0:1789760750.785412:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5624 00000100:00100000:3.0:1789760750.785422:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729150716224 time 729203252030 state 0x2 ref 0 00000100:00100000:0.0:1789760750.792749:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760750.793682:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715584 00000100:00100000:1.0:1789760750.793697:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715584 00000100:00100000:1.0:1789760750.793789:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729150716224 time 729203252030 state 0x0 ref 1 00000100:00100000:1.0:1789760750.793803:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729203512446 time 729203252030 state 0x1 ref 1 00000100:00100000:1.0:1789760750.794641:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729203512446 time 729259122394 state 0x0 ref 1 00000100:00100000:0.0:1789760750.794732:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515715584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760750.794783:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22921216, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:0.0:1789760750.794792:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.794809:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.794814:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.794821:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.794844:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.847262:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972913 is committed 00002000:00010000:0.0:1789760750.847348:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22921216 -> 22925312 00010000:00010000:0.0:1789760750.847364:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.847376:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.847386:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.847394:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760750.847399:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.847414:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.847435:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.847442:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1eb6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.847569:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515715584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52908us (54942us total) trans 4294972913 rc 0/0 00000100:00100000:0.0:1789760750.847579:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5625 00000100:00100000:0.0:1789760750.847589:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729203512446 time 729259122394 state 0x2 ref 0 00000100:00100000:2.0:1789760750.854156:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.854511:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715712 00000100:00100000:3.0:1789760750.854521:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715712 00000100:00100000:3.0:1789760750.854587:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729203512446 time 729259122394 state 0x0 ref 1 00000100:00100000:3.0:1789760750.854603:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729259382810 time 729259122394 state 0x1 ref 1 00000100:00100000:3.0:1789760750.854636:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729259382810 time 729319924309 state 0x0 ref 1 00000100:00100000:3.0:1789760750.854649:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515715712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.854696:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22925312, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xb007 00010000:00010000:3.0:1789760750.854736:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.854752:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.854757:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.854763:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.854777:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.893276:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972914 is committed 00002000:00010000:0.0:1789760750.893383:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22925312 -> 22929408 00002000:00010000:0.0:1789760750.893387:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760750 -> 1789760751 00002000:00010000:0.0:1789760750.893390:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760750 -> 1789760751 00010000:00010000:0.0:1789760750.893402:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.893414:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.893423:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.893435:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760750.893439:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.893448:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.893467:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.893475:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.893572:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515715712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38920us (39419us total) trans 4294972914 rc 0/0 00000100:00100000:0.0:1789760750.893581:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5626 00000100:00100000:0.0:1789760750.893591:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729259382810 time 729319924309 state 0x2 ref 0 00000100:00100000:2.0:1789760750.898123:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760750.898236:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715840 00000100:00100000:1.0:1789760750.898246:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715840 00000100:00100000:1.0:1789760750.898295:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729259382810 time 729319924309 state 0x0 ref 1 00000100:00100000:1.0:1789760750.898311:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729320184725 time 729319924309 state 0x1 ref 1 00000100:00100000:1.0:1789760750.898367:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729320184725 time 729363655957 state 0x0 ref 1 00000100:00100000:1.0:1789760750.898381:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515715840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760750.898430:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22929408, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:1.0:1789760750.898438:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.898454:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760750.898459:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.898465:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.898479:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.911529:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760750.924703:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22929408 -> 22933504 00010000:00010000:1.0:1789760750.924755:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.924768:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.924778:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.924786:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760750.924791:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.924801:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.924819:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.924826:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eb7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.924928:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515715840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26545us (26811us total) trans 4294972915 rc 0/0 00000100:00100000:1.0:1789760750.924939:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5627 00000100:00100000:1.0:1789760750.924948:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729320184725 time 729363655957 state 0x2 ref 0 00000001:00080000:3.0:1789760750.928104:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972915 is committed 00000100:00100000:3.0:1789760750.928970:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760750.929091:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515715968 00000100:00100000:2.0:1789760750.929100:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515715968 00000100:00100000:2.0:1789760750.929149:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729320184725 time 729363655957 state 0x0 ref 1 00000100:00100000:2.0:1789760750.929164:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729363916373 time 729363655957 state 0x1 ref 1 00000100:00100000:2.0:1789760750.929193:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729363916373 time 729394482601 state 0x0 ref 1 00000100:00100000:2.0:1789760750.929204:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515715968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760750.929260:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22933504, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760750.929269:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.929284:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.929288:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.929295:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.929309:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.954669:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972916 is committed 00002000:00010000:0.0:1789760750.954835:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22933504 -> 22937600 00010000:00010000:0.0:1789760750.954850:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.954862:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.954878:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.954886:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760750.954890:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.954900:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.954930:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.954938:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1eb83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.955029:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515715968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25821us (26060us total) trans 4294972916 rc 0/0 00000100:00100000:0.0:1789760750.955040:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5628 00000100:00100000:0.0:1789760750.955049:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729363916373 time 729394482601 state 0x2 ref 0 00000100:00100000:0.0:1789760750.959015:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760750.959105:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716096 00000100:00100000:2.0:1789760750.959115:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716096 00000100:00100000:2.0:1789760750.959178:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729363916373 time 729394482601 state 0x0 ref 1 00000100:00100000:2.0:1789760750.959193:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729394743017 time 729394482601 state 0x1 ref 1 00000100:00100000:2.0:1789760750.959222:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729394743017 time 729424511172 state 0x0 ref 1 00000100:00100000:2.0:1789760750.959234:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515716096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760750.959282:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22937600, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760750.959291:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.959305:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.959311:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.959317:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.959331:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.983766:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22937600 -> 22941696 00010000:00010000:2.0:1789760750.983786:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983801:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983811:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983818:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760750.983823:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983903:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983936:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.983945:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1eb8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.984030:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515716096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24794us (25020us total) trans 4294972917 rc 0/0 00000100:00100000:2.0:1789760750.984041:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5629 00000100:00100000:2.0:1789760750.984050:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729394743017 time 729424511172 state 0x2 ref 0 00000001:00080000:3.0:1789760750.984127:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972917 is committed 00000100:00100000:2.0:1789760750.988218:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760750.988359:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716224 00000100:00100000:3.0:1789760750.988369:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716224 00000100:00100000:3.0:1789760750.988422:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729394743017 time 729424511172 state 0x0 ref 1 00000100:00100000:3.0:1789760750.988436:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729424771588 time 729424511172 state 0x1 ref 1 00000100:00100000:3.0:1789760750.988469:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729424771588 time 729453757993 state 0x0 ref 1 00000100:00100000:3.0:1789760750.988483:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515716224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760750.988530:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22941696, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:3.0:1789760750.988539:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.988555:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.988571:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.988578:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.988592:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760751.017053:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22941696 -> 22945792 00010000:00010000:1.0:1789760751.017074:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.017087:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.017095:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.017103:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.017109:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.017122:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.017141:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.017149:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1eb91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.017261:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515716224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28776us (29047us total) trans 4294972918 rc 0/0 00000100:00100000:1.0:1789760751.017272:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5630 00000100:00100000:1.0:1789760751.017282:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729424771588 time 729453757993 state 0x2 ref 0 00000001:00080000:2.0:1789760751.017683:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972918 is committed 00000100:00100000:0.0:1789760751.021616:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760751.021698:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716352 00000100:00100000:3.0:1789760751.021743:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716352 00000100:00100000:3.0:1789760751.021798:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729424771588 time 729453757993 state 0x0 ref 1 00000100:00100000:3.0:1789760751.021814:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729454018409 time 729453757993 state 0x1 ref 1 00000100:00100000:3.0:1789760751.021838:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729454018409 time 729487127184 state 0x0 ref 1 00000100:00100000:3.0:1789760751.021850:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760751.025526:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22945792, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:3.0:1789760751.025544:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.025568:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760751.025573:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.025580:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.025595:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.080749:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972919 is committed 00002000:00010000:0.0:1789760751.080869:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22945792 -> 22949888 00010000:00010000:0.0:1789760751.080884:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.080897:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.080906:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.080923:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.080927:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.080936:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.080953:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.080960:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1eb98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.081051:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59199us (59441us total) trans 4294972919 rc 0/0 00000100:00100000:0.0:1789760751.081061:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5631 00000100:00100000:0.0:1789760751.081070:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729454018409 time 729487127184 state 0x2 ref 0 00000100:00100000:0.0:1789760751.087744:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760751.088897:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716480 00000100:00100000:3.0:1789760751.088920:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716480 00000100:00100000:3.0:1789760751.088980:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729454018409 time 729487127184 state 0x0 ref 1 00000100:00100000:3.0:1789760751.088995:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729487387600 time 729487127184 state 0x1 ref 1 00000100:00100000:3.0:1789760751.089272:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729487387600 time 729554558417 state 0x0 ref 1 00000100:00100000:3.0:1789760751.089288:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760751.089336:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22949888, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:3.0:1789760751.089346:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.089361:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760751.089365:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.089371:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.089386:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.148692:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972920 is committed 00002000:00010000:0.0:1789760751.148966:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22949888 -> 22953984 00010000:00010000:0.0:1789760751.148982:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.148994:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.149004:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.149012:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.149017:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.149027:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.149048:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.149057:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1eb9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.149150:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59860us (61448us total) trans 4294972920 rc 0/0 00000100:00100000:0.0:1789760751.149160:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5632 00000100:00100000:0.0:1789760751.149170:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729487387600 time 729554558417 state 0x2 ref 0 00000100:00100000:3.0:1789760751.152310:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.152405:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716608 00000100:00100000:1.0:1789760751.152414:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716608 00000100:00100000:1.0:1789760751.152474:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729487387600 time 729554558417 state 0x0 ref 1 00000100:00100000:1.0:1789760751.152489:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729554818833 time 729554558417 state 0x1 ref 1 00000100:00100000:1.0:1789760751.152518:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729554818833 time 729617808059 state 0x0 ref 1 00000100:00100000:1.0:1789760751.152534:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515716608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.152588:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22953984, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:1.0:1789760751.152598:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.152613:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760751.152617:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.152623:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.152636:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.205179:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972921 is committed 00002000:00010000:1.0:1789760751.208512:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22953984 -> 22958080 00010000:00010000:1.0:1789760751.208533:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208545:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208629:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208638:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.208643:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208653:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208679:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208688:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eba6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.208931:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515716608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56393us (56624us total) trans 4294972921 rc 0/0 00000100:00100000:1.0:1789760751.208943:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5633 00000100:00100000:1.0:1789760751.208953:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729554818833 time 729617808059 state 0x2 ref 0 00000100:00100000:3.0:1789760751.226092:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760751.228324:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716736 00000100:00100000:0.0:1789760751.228339:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716736 00000100:00100000:2.0:1789760751.230476:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729554818833 time 729617808059 state 0x0 ref 1 00000100:00100000:2.0:1789760751.230495:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729618068475 time 729617808059 state 0x1 ref 1 00000100:00100000:2.0:1789760751.230528:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729618068475 time 729695816920 state 0x0 ref 1 00000100:00100000:2.0:1789760751.230543:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760751.230593:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22958080, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760751.230602:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.230617:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760751.230622:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.230629:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.230644:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760751.320591:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22958080 -> 22962176 00010000:00010000:0.0:1789760751.320611:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.320625:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.320634:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.320642:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.320647:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.320656:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.320676:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.320683:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ebad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.320810:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515716736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 90266us (94724us total) trans 4294972922 rc 0/0 00000100:00100000:0.0:1789760751.320820:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5634 00000100:00100000:0.0:1789760751.320840:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729618068475 time 729695816920 state 0x2 ref 0 00000001:00080000:0.0:1789760751.320876:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972922 is committed 00000100:00100000:0.0:1789760751.325497:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760751.325602:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716864 00000100:00100000:2.0:1789760751.325611:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716864 00000100:00100000:2.0:1789760751.325667:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729618068475 time 729695816920 state 0x0 ref 1 00000100:00100000:2.0:1789760751.325683:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729696077336 time 729695816920 state 0x1 ref 1 00000100:00100000:2.0:1789760751.325755:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729696077336 time 729791033565 state 0x0 ref 1 00000100:00100000:2.0:1789760751.325768:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760751.325818:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22962176, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760751.325834:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.325850:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760751.325854:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.325860:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.325883:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.360465:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972923 is committed 00002000:00010000:2.0:1789760751.360547:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22962176 -> 22966272 00010000:00010000:2.0:1789760751.360562:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.360579:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.360588:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.360595:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760751.360600:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.360609:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.360628:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.360636:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1ebb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.360796:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515716864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35025us (35302us total) trans 4294972923 rc 0/0 00000100:00100000:2.0:1789760751.360808:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5635 00000100:00100000:2.0:1789760751.360818:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729696077336 time 729791033565 state 0x2 ref 0 00000100:00100000:0.0:1789760751.364643:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760751.365352:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515716992 00000100:00100000:3.0:1789760751.365365:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515716992 00000100:00100000:3.0:1789760751.365420:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729696077336 time 729791033565 state 0x0 ref 1 00000100:00100000:3.0:1789760751.365435:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729791293981 time 729791033565 state 0x1 ref 1 00000100:00100000:3.0:1789760751.365474:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729791293981 time 729830763259 state 0x0 ref 1 00000100:00100000:3.0:1789760751.365487:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515716992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760751.365532:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22966272, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:3.0:1789760751.365542:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.365891:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760751.365897:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.365903:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.365928:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.394247:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972924 is committed 00002000:00010000:0.0:1789760751.394335:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22966272 -> 22970368 00010000:00010000:0.0:1789760751.394350:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.394362:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.394371:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.394379:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.394384:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.394394:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.394412:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.394426:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ebbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.394626:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515716992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29136us (29989us total) trans 4294972924 rc 0/0 00000100:00100000:0.0:1789760751.394636:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5636 00000100:00100000:0.0:1789760751.394646:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729791293981 time 729830763259 state 0x2 ref 0 00000100:00100000:3.0:1789760751.399156:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760751.399242:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717120 00000100:00100000:0.0:1789760751.399253:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717120 00000100:00100000:0.0:1789760751.399305:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729791293981 time 729830763259 state 0x0 ref 1 00000100:00100000:0.0:1789760751.399320:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729831023675 time 729830763259 state 0x1 ref 1 00000100:00100000:0.0:1789760751.399354:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729831023675 time 729864642756 state 0x0 ref 1 00000100:00100000:0.0:1789760751.399365:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515717120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760751.399409:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22970368, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:0.0:1789760751.399418:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.399434:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760751.399438:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.399444:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.399460:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.439296:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972925 is committed 00002000:00010000:2.0:1789760751.439508:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22970368 -> 22974464 00010000:00010000:2.0:1789760751.439526:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.439539:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.439615:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.439624:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760751.439629:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.439639:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.439658:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.439666:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.440576:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515717120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41194us (41411us total) trans 4294972925 rc 0/0 00000100:00100000:2.0:1789760751.440589:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5637 00000100:00100000:2.0:1789760751.440599:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729831023675 time 729864642756 state 0x2 ref 0 00000100:00100000:2.0:1789760751.444426:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760751.444663:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717248 00000100:00100000:0.0:1789760751.444673:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717248 00000100:00100000:0.0:1789760751.444762:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729831023675 time 729864642756 state 0x0 ref 1 00000100:00100000:0.0:1789760751.444778:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729864903172 time 729864642756 state 0x1 ref 1 00000100:00100000:0.0:1789760751.444806:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729864903172 time 729910095470 state 0x0 ref 1 00000100:00100000:0.0:1789760751.444819:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515717248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760751.444881:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22974464, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:0.0:1789760751.444889:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.444905:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760751.444920:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.444926:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.444941:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.474173:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972926 is committed 00002000:00010000:1.0:1789760751.474276:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22974464 -> 22978560 00010000:00010000:1.0:1789760751.474292:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.474305:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.474314:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.474321:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.474326:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.474340:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.474359:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.474366:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1ebc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.474454:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515717248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29632us (30032us total) trans 4294972926 rc 0/0 00000100:00100000:1.0:1789760751.474464:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5638 00000100:00100000:1.0:1789760751.474473:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729864903172 time 729910095470 state 0x2 ref 0 00000100:00100000:0.0:1789760751.479358:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.479435:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717376 00000100:00100000:1.0:1789760751.479444:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717376 00000100:00100000:1.0:1789760751.479495:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729864903172 time 729910095470 state 0x0 ref 1 00000100:00100000:1.0:1789760751.479510:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729910355886 time 729910095470 state 0x1 ref 1 00000100:00100000:1.0:1789760751.479540:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729910355886 time 729944828880 state 0x0 ref 1 00000100:00100000:1.0:1789760751.479552:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515717376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.479599:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22978560, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:1.0:1789760751.479609:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.479624:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760751.479627:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.479633:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.479646:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.515355:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972927 is committed 00002000:00010000:1.0:1789760751.515465:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22978560 -> 22982656 00010000:00010000:1.0:1789760751.515482:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.515495:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.515503:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.515510:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.515515:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.515525:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.515544:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.516625:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.516783:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515717376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37228us (37429us total) trans 4294972927 rc 0/0 00000100:00100000:1.0:1789760751.516794:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5639 00000100:00100000:1.0:1789760751.516805:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729910355886 time 729944828880 state 0x2 ref 0 00000100:00100000:3.0:1789760751.526183:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760751.528039:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717504 00000100:00100000:2.0:1789760751.528053:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717504 00000100:00100000:2.0:1789760751.528112:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729910355886 time 729944828880 state 0x0 ref 1 00000100:00100000:2.0:1789760751.528127:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729945089296 time 729944828880 state 0x1 ref 1 00000100:00100000:2.0:1789760751.528166:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729945089296 time 729993454592 state 0x0 ref 1 00000100:00100000:2.0:1789760751.528180:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515717504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760751.528243:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22982656, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760751.528252:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.528277:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760751.528282:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.528289:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.528302:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760751.571609:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22982656 -> 22986752 00010000:00010000:0.0:1789760751.571631:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.571645:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.571654:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.571662:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.571668:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.571677:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.571698:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.571727:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ebd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.571838:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515717504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43649us (45930us total) trans 4294972928 rc 0/0 00000100:00100000:0.0:1789760751.571849:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5640 00000100:00100000:0.0:1789760751.571857:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729945089296 time 729993454592 state 0x2 ref 0 00000001:00080000:1.0:1789760751.572340:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972928 is committed 00000100:00100000:0.0:1789760751.579392:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760751.579514:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717632 00000100:00100000:2.0:1789760751.579525:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717632 00000100:00100000:2.0:1789760751.579581:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729945089296 time 729993454592 state 0x0 ref 1 00000100:00100000:2.0:1789760751.579597:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729993715008 time 729993454592 state 0x1 ref 1 00000100:00100000:2.0:1789760751.579673:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729993715008 time 730044961176 state 0x0 ref 1 00000100:00100000:2.0:1789760751.579686:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515717632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760751.579763:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22986752, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760751.579772:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.579787:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760751.579792:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.579798:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.579812:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760751.608059:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22986752 -> 22990848 00010000:00010000:0.0:1789760751.608080:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608093:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608103:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608110:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.608115:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608125:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608145:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.608154:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1ebde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.608318:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515717632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28629us (28930us total) trans 4294972929 rc 0/0 00000100:00100000:0.0:1789760751.608333:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5641 00000100:00100000:0.0:1789760751.608342:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729993715008 time 730044961176 state 0x2 ref 0 00000001:00080000:1.0:1789760751.608424:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972929 is committed 00000100:00100000:3.0:1789760751.613961:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760751.614249:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717760 00000100:00100000:0.0:1789760751.614260:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717760 00000100:00100000:0.0:1789760751.614314:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 729993715008 time 730044961176 state 0x0 ref 1 00000100:00100000:0.0:1789760751.614328:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730045221592 time 730044961176 state 0x1 ref 1 00000100:00100000:0.0:1789760751.614359:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730045221592 time 730079647890 state 0x0 ref 1 00000100:00100000:0.0:1789760751.614372:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515717760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760751.614419:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22990848, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:0.0:1789760751.614428:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.614443:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760751.614447:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.614454:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.614468:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.652527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972930 is committed 00002000:00010000:1.0:1789760751.652639:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22990848 -> 22994944 00010000:00010000:1.0:1789760751.652657:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.652672:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.652682:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.652689:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.652695:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.652725:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.652747:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.652756:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ebe5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.652858:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515717760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38483us (38901us total) trans 4294972930 rc 0/0 00000100:00100000:1.0:1789760751.652867:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5642 00000100:00100000:1.0:1789760751.652877:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730045221592 time 730079647890 state 0x2 ref 0 00000100:00100000:3.0:1789760751.663112:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760751.663219:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515717888 00000100:00100000:0.0:1789760751.663229:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515717888 00000100:00100000:0.0:1789760751.663285:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730045221592 time 730079647890 state 0x0 ref 1 00000100:00100000:0.0:1789760751.663301:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730079908306 time 730079647890 state 0x1 ref 1 00000100:00100000:0.0:1789760751.663335:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730079908306 time 730128624276 state 0x0 ref 1 00000100:00100000:0.0:1789760751.663349:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515717888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760751.663405:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22994944, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:0.0:1789760751.663415:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.663429:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760751.663434:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.663441:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.663454:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760751.755332:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22994944 -> 22999040 00010000:00010000:0.0:1789760751.755370:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.755383:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.755392:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.755400:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760751.755405:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.755414:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.755434:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.755442:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ebec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.755566:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515717888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 92213us (92459us total) trans 4294972931 rc 0/0 00000100:00100000:0.0:1789760751.755577:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5643 00000100:00100000:0.0:1789760751.755587:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730079908306 time 730128624276 state 0x2 ref 0 00000001:00080000:1.0:1789760751.755654:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972931 is committed 00000100:00100000:2.0:1789760751.762527:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.762616:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718016 00000100:00100000:1.0:1789760751.762626:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718016 00000100:00100000:1.0:1789760751.762682:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730079908306 time 730128624276 state 0x0 ref 1 00000100:00100000:1.0:1789760751.762697:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730128884692 time 730128624276 state 0x1 ref 1 00000100:00100000:1.0:1789760751.762760:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730128884692 time 730228048362 state 0x0 ref 1 00000100:00100000:1.0:1789760751.762776:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515718016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.762826:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 22999040, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:1.0:1789760751.763002:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.763023:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.763027:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.763034:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.763049:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.790705:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972932 is committed 00002000:00010000:1.0:1789760751.790994:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 22999040 -> 23003136 00010000:00010000:1.0:1789760751.791010:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.791022:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.791030:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.791037:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.791042:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.791053:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.791071:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.791079:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ebf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.791208:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515718016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28431us (28685us total) trans 4294972932 rc 0/0 00000100:00100000:1.0:1789760751.791219:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5644 00000100:00100000:1.0:1789760751.791229:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730128884692 time 730228048362 state 0x2 ref 0 00000100:00100000:2.0:1789760751.795780:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.795878:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718144 00000100:00100000:1.0:1789760751.795888:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718144 00000100:00100000:1.0:1789760751.795951:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730128884692 time 730228048362 state 0x0 ref 1 00000100:00100000:1.0:1789760751.795966:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730228308778 time 730228048362 state 0x1 ref 1 00000100:00100000:1.0:1789760751.795987:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730228308778 time 730261276618 state 0x0 ref 1 00000100:00100000:1.0:1789760751.795999:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515718144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.796046:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23003136, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:1.0:1789760751.796055:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.796070:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.796075:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.796081:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.796100:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760751.822987:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23003136 -> 23007232 00010000:00010000:2.0:1789760751.823009:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.823021:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.823037:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.823045:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760751.823050:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.823059:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.823078:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.823086:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ebfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.823181:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515718144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27177us (27405us total) trans 4294972933 rc 0/0 00000100:00100000:2.0:1789760751.823191:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5645 00000100:00100000:2.0:1789760751.823201:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730228308778 time 730261276618 state 0x2 ref 0 00000100:00100000:1.0:1789760751.826585:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760751.826737:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718272 00000100:00100000:2.0:1789760751.826748:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718272 00000100:00100000:2.0:1789760751.826797:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730228308778 time 730261276618 state 0x0 ref 1 00000100:00100000:2.0:1789760751.826814:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730261537034 time 730261276618 state 0x1 ref 1 00000100:00100000:2.0:1789760751.826853:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730261537034 time 730292141315 state 0x0 ref 1 00000100:00100000:2.0:1789760751.826866:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515718272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760751.826921:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23007232, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xb007 00010000:00010000:2.0:1789760751.826930:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.826944:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760751.826949:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.826955:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.826970:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.828176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972933 is committed 00000001:00080000:1.0:1789760751.867977:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972934 is committed 00002000:00010000:3.0:1789760751.868148:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23007232 -> 23011328 00002000:00010000:3.0:1789760751.868153:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:3.0:1789760751.868156:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00010000:00010000:3.0:1789760751.868169:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.868181:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.868190:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.868198:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760751.868203:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.868212:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.868235:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.868243:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ec01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.868328:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515718272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41458us (41745us total) trans 4294972934 rc 0/0 00000100:00100000:3.0:1789760751.868339:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5646 00000100:00100000:3.0:1789760751.868349:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730261537034 time 730292141315 state 0x2 ref 0 00000100:00100000:0.0:1789760751.872110:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.872188:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718400 00000100:00100000:1.0:1789760751.872198:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718400 00000100:00100000:1.0:1789760751.872246:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730261537034 time 730292141315 state 0x0 ref 1 00000100:00100000:1.0:1789760751.872261:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730292401731 time 730292141315 state 0x1 ref 1 00000100:00100000:1.0:1789760751.872291:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730292401731 time 730337579808 state 0x0 ref 1 00000100:00100000:1.0:1789760751.872304:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515718400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.872348:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23011328, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:1.0:1789760751.872356:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.872371:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.872376:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.872382:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.872397:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.907201:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972935 is committed 00002000:00010000:1.0:1789760751.907325:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23011328 -> 23015424 00010000:00010000:1.0:1789760751.907342:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.907356:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.907367:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.907375:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.907380:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.907390:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.907412:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.907421:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.907692:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515718400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35384us (35585us total) trans 4294972935 rc 0/0 00000100:00100000:1.0:1789760751.907703:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5647 00000100:00100000:1.0:1789760751.907750:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730292401731 time 730337579808 state 0x2 ref 0 00000100:00100000:0.0:1789760751.912224:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.912319:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718528 00000100:00100000:1.0:1789760751.912329:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718528 00000100:00100000:1.0:1789760751.912385:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730292401731 time 730337579808 state 0x0 ref 1 00000100:00100000:1.0:1789760751.912400:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730337840224 time 730337579808 state 0x1 ref 1 00000100:00100000:1.0:1789760751.912464:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730337840224 time 730377752472 state 0x0 ref 1 00000100:00100000:1.0:1789760751.912486:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515718528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.912538:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23015424, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:1.0:1789760751.912664:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.912680:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.912685:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.912692:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.912735:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.933988:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760751.940281:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972936 is committed 00002000:00010000:1.0:1789760751.940614:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23015424 -> 23019520 00010000:00010000:1.0:1789760751.940631:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.940644:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.940654:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.940662:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.940667:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.940676:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.940695:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.940702:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.940955:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515718528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28474us (28735us total) trans 4294972936 rc 0/0 00000100:00100000:1.0:1789760751.940966:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5648 00000100:00100000:1.0:1789760751.940976:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730337840224 time 730377752472 state 0x2 ref 0 00000100:00100000:2.0:1789760751.943477:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.943583:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718656 00000100:00100000:1.0:1789760751.943593:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718656 00000100:00100000:1.0:1789760751.943642:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730337840224 time 730377752472 state 0x0 ref 1 00000100:00100000:1.0:1789760751.943656:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730378012888 time 730377752472 state 0x1 ref 1 00000100:00100000:1.0:1789760751.943686:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730378012888 time 730408974627 state 0x0 ref 1 00000100:00100000:1.0:1789760751.943700:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515718656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.943772:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23019520, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:1.0:1789760751.943780:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.943794:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.943810:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.943817:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.943831:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760751.980091:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23019520 -> 23023616 00010000:00010000:1.0:1789760751.980112:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.980125:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.980141:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.980150:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760751.980154:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.980164:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.980193:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.980200:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.980290:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515718656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36589us (36816us total) trans 4294972937 rc 0/0 00000100:00100000:1.0:1789760751.980300:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5649 00000100:00100000:1.0:1789760751.980310:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730378012888 time 730408974627 state 0x2 ref 0 00000001:00080000:3.0:1789760751.980437:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972937 is committed 00000100:00100000:0.0:1789760751.984893:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760751.985791:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718784 00000100:00100000:1.0:1789760751.985803:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718784 00000100:00100000:1.0:1789760751.985866:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730378012888 time 730408974627 state 0x0 ref 1 00000100:00100000:1.0:1789760751.985881:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730409235043 time 730408974627 state 0x1 ref 1 00000100:00100000:1.0:1789760751.985924:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730409235043 time 730451212655 state 0x0 ref 1 00000100:00100000:1.0:1789760751.985936:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515718784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760751.985979:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23023616, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:1.0:1789760751.985988:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.986003:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.986008:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.986014:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.986029:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.015877:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23023616 -> 23027712 00010000:00010000:0.0:1789760752.015901:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.015929:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.015938:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.015946:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.015951:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.015960:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.015981:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.015993:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ec1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.016017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972938 is committed 00000100:00100000:0.0:1789760752.016078:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515718784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30140us (31190us total) trans 4294972938 rc 0/0 00000100:00100000:0.0:1789760752.016089:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5650 00000100:00100000:0.0:1789760752.016098:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730409235043 time 730451212655 state 0x2 ref 0 00000100:00100000:0.0:1789760752.021086:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760752.022430:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515718912 00000100:00100000:2.0:1789760752.022443:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515718912 00000100:00100000:2.0:1789760752.022519:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730409235043 time 730451212655 state 0x0 ref 1 00000100:00100000:2.0:1789760752.022533:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730451473071 time 730451212655 state 0x1 ref 1 00000100:00100000:2.0:1789760752.022567:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730451473071 time 730487856956 state 0x0 ref 1 00000100:00100000:2.0:1789760752.022582:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515718912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760752.022628:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23027712, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:2.0:1789760752.022636:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.022650:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760752.022654:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.022661:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.022673:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760752.022944:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760752.055395:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972939 is committed 00002000:00010000:1.0:1789760752.055692:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23027712 -> 23031808 00010000:00010000:1.0:1789760752.055739:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.055752:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.055763:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.055771:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760752.055777:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.055787:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.055808:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.055817:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.055924:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515718912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33339us (34842us total) trans 4294972939 rc 0/0 00000100:00100000:1.0:1789760752.055934:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5651 00000100:00100000:1.0:1789760752.055943:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730451473071 time 730487856956 state 0x2 ref 0 00000100:00100000:2.0:1789760752.060086:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.060269:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719040 00000100:00100000:3.0:1789760752.060280:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719040 00000100:00100000:3.0:1789760752.060349:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730451473071 time 730487856956 state 0x0 ref 1 00000100:00100000:3.0:1789760752.060364:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730488117372 time 730487856956 state 0x1 ref 1 00000100:00100000:3.0:1789760752.060394:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730488117372 time 730525682776 state 0x0 ref 1 00000100:00100000:3.0:1789760752.060406:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515719040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.060463:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23031808, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:3.0:1789760752.060472:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.060489:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760752.060494:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.060500:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.060513:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.112273:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972940 is committed 00002000:00010000:1.0:1789760752.112347:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23031808 -> 23035904 00010000:00010000:1.0:1789760752.112364:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.112376:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.112386:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.112399:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760752.112405:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.112414:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.112434:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.112443:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ec2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.112532:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515719040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52123us (52451us total) trans 4294972940 rc 0/0 00000100:00100000:1.0:1789760752.112541:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5652 00000100:00100000:1.0:1789760752.112618:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730488117372 time 730525682776 state 0x2 ref 0 00000100:00100000:0.0:1789760752.123559:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760752.123830:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719168 00000100:00100000:2.0:1789760752.123840:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719168 00000100:00100000:2.0:1789760752.123899:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730488117372 time 730525682776 state 0x0 ref 1 00000100:00100000:2.0:1789760752.123927:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730525943192 time 730525682776 state 0x1 ref 1 00000100:00100000:2.0:1789760752.123957:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730525943192 time 730589246278 state 0x0 ref 1 00000100:00100000:2.0:1789760752.123971:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515719168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760752.124022:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23035904, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:2.0:1789760752.124032:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.124050:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760752.124055:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.124062:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.124077:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.132125:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544413952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.132205:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760752.132298:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544413952 00000100:00100000:2.0:1789760752.132306:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544413952 00000100:00100000:2.0:1789760752.132352:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1789760752.132361:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909353450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544413952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760752.132780:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544414080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760752.132864:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760752.132893:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760752.132899:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544414208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760752.132933:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760752.132962:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414080 00000100:00100000:3.0:1789760752.132966:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414080 00000100:00100000:3.0:1789760752.132993:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414208 00000100:00100000:3.0:1789760752.132996:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414208 00000100:00100000:3.0:1789760752.133023:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1789760752.133030:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544414080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760752.133069:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544414080:12345-0@lo:400:kworker.0 Request processed in 39us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760752.133077:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1789760752.133093:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1789760752.133098:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544414208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760752.133127:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544414208:12345-0@lo:400:kworker.0 Request processed in 29us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760752.133133:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00080000:3.0:1789760752.133151:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760752.133328:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544414208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760752.133338:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544414080:0@lo:400:kworker.0 00000100:00100000:2.0:1789760752.138025:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909353450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544413952:12345-0@lo:400:kworker.0 Request processed in 5654us (5808us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760752.138041:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1789760752.138109:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93b669c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544414336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760752.138161:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760752.138865:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760752.138878:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804df0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544414592:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.138894:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544413952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760752.138928:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.138990:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804df2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544414464:0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.139026:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.139080:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414464 00000100:00100000:1.0:1789760752.139087:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414464 00000100:00100000:1.0:1789760752.139221:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760752.139231:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544414464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.139295:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c030700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544414464:12345-0@lo:400:kworker.0 Request processed in 64us (267us total) trans 0 rc 0/0 00000100:00080000:3.0:1789760752.139300:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760752.139305:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760752.139330:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414336 00000100:00100000:1.0:1789760752.139334:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414336 00000100:00100000:2.0:1789760752.139368:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804df2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544414464:0@lo:400:kworker.0 00000100:00080000:3.0:1789760752.139421:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760752.139524:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414592 00000100:00100000:1.0:1789760752.139529:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414592 00000100:00100000:1.0:1789760752.139574:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:1.0:1789760752.139582:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544414336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760752.139623:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544414336:12345-0@lo:400:kworker.0 Request processed in 41us (1462us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.139630:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:2.0:1789760752.139639:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789760752.139650:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544414592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760752.139694:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544414592:12345-0@lo:400:kworker.0 Request processed in 46us (765us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760752.139699:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789760752.139750:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804df0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544414592:0@lo:400:kworker.0 00000100:00080000:0.0:1789760752.139806:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760752.140257:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93b669c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544414336:0@lo:400:kworker.0 00000001:00080000:3.0:1789760752.179574:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972941 is committed 00002000:00010000:2.0:1789760752.179746:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23035904 -> 23040000 00010000:00010000:2.0:1789760752.179761:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.179774:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.179784:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.179791:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760752.179796:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.179805:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.179885:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.179893:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1ec32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.179986:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515719168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56012us (56442us total) trans 4294972941 rc 0/0 00000100:00100000:2.0:1789760752.179998:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5653 00000100:00100000:2.0:1789760752.180009:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730525943192 time 730589246278 state 0x2 ref 0 00000100:00100000:2.0:1789760752.183503:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.183667:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719296 00000100:00100000:3.0:1789760752.183676:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719296 00000100:00100000:3.0:1789760752.183751:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730525943192 time 730589246278 state 0x0 ref 1 00000100:00100000:3.0:1789760752.183767:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730589506694 time 730589246278 state 0x1 ref 1 00000100:00100000:3.0:1789760752.183792:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730589506694 time 730649081092 state 0x0 ref 1 00000100:00100000:3.0:1789760752.183804:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515719296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.183862:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23040000, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:3.0:1789760752.183871:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.183888:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760752.183893:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.183899:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.183928:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.210813:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972942 is committed 00002000:00010000:3.0:1789760752.210985:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23040000 -> 23044096 00010000:00010000:3.0:1789760752.211001:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.211013:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.211022:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.211030:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760752.211035:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.211044:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.211062:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.211071:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.211167:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515719296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27359us (27668us total) trans 4294972942 rc 0/0 00000100:00100000:3.0:1789760752.211177:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5654 00000100:00100000:3.0:1789760752.211186:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730589506694 time 730649081092 state 0x2 ref 0 00000100:00100000:2.0:1789760752.215076:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.215310:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719424 00000100:00100000:3.0:1789760752.215320:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719424 00000100:00100000:3.0:1789760752.215372:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730589506694 time 730649081092 state 0x0 ref 1 00000100:00100000:3.0:1789760752.215388:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730649341508 time 730649081092 state 0x1 ref 1 00000100:00100000:3.0:1789760752.215418:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730649341508 time 730680707251 state 0x0 ref 1 00000100:00100000:3.0:1789760752.215432:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515719424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.215477:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23044096, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:3.0:1789760752.215485:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.215501:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760752.215506:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.215512:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.215527:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.248477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972943 is committed 00002000:00010000:0.0:1789760752.248651:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23044096 -> 23048192 00010000:00010000:0.0:1789760752.248667:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.248679:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.248687:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.248694:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.248699:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.248730:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.248750:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.248759:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1ec40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.248911:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515719424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33467us (33829us total) trans 4294972943 rc 0/0 00000100:00100000:0.0:1789760752.248922:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5655 00000100:00100000:0.0:1789760752.248932:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730649341508 time 730680707251 state 0x2 ref 0 00000100:00100000:0.0:1789760752.252060:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.252170:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719552 00000100:00100000:1.0:1789760752.252179:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719552 00000100:00100000:1.0:1789760752.252228:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730649341508 time 730680707251 state 0x0 ref 1 00000100:00100000:1.0:1789760752.252244:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730680967667 time 730680707251 state 0x1 ref 1 00000100:00100000:1.0:1789760752.252276:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730680967667 time 730717565541 state 0x0 ref 1 00000100:00100000:1.0:1789760752.252289:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515719552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760752.252337:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23048192, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:1.0:1789760752.252346:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.252363:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760752.252367:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.252373:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.252387:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.274076:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972944 is committed 00002000:00010000:0.0:1789760752.274279:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23048192 -> 23052288 00010000:00010000:0.0:1789760752.274295:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.274308:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.274316:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.274323:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.274328:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.274337:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.274357:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.274370:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ec47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.274457:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515719552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22164us (22402us total) trans 4294972944 rc 0/0 00000100:00100000:0.0:1789760752.274468:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5656 00000100:00100000:0.0:1789760752.274479:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730680967667 time 730717565541 state 0x2 ref 0 00000100:00100000:2.0:1789760752.277524:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.277637:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719680 00000100:00100000:1.0:1789760752.277647:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719680 00000100:00100000:1.0:1789760752.277705:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730680967667 time 730717565541 state 0x0 ref 1 00000100:00100000:1.0:1789760752.277739:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730717825957 time 730717565541 state 0x1 ref 1 00000100:00100000:1.0:1789760752.277768:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730717825957 time 730743057337 state 0x0 ref 1 00000100:00100000:1.0:1789760752.277780:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515719680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760752.277933:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23052288, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:1.0:1789760752.277943:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.277959:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760752.277964:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.277970:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.277989:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.310387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972945 is committed 00002000:00010000:2.0:1789760752.310937:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23052288 -> 23056384 00010000:00010000:2.0:1789760752.310954:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.311030:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.311041:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.311047:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760752.311051:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.311061:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.311080:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.311088:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.311185:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515719680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33403us (33666us total) trans 4294972945 rc 0/0 00000100:00100000:2.0:1789760752.311196:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5657 00000100:00100000:2.0:1789760752.311206:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730717825957 time 730743057337 state 0x2 ref 0 00000100:00100000:2.0:1789760752.317310:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760752.317389:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719808 00000100:00100000:0.0:1789760752.317399:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719808 00000100:00100000:0.0:1789760752.317449:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730717825957 time 730743057337 state 0x0 ref 1 00000100:00100000:0.0:1789760752.317463:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730743317753 time 730743057337 state 0x1 ref 1 00000100:00100000:0.0:1789760752.317491:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730743317753 time 730782781004 state 0x0 ref 1 00000100:00100000:0.0:1789760752.317503:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515719808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760752.317547:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23056384, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:0.0:1789760752.317556:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.317571:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760752.317575:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.317580:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.317599:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.352407:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972946 is committed 00002000:00010000:1.0:1789760752.352494:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23056384 -> 23060480 00010000:00010000:1.0:1789760752.352510:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.352523:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.352533:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.352542:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760752.352547:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.352557:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.352578:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.352587:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ec55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.352688:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515719808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35182us (35382us total) trans 4294972946 rc 0/0 00000100:00100000:1.0:1789760752.352700:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5658 00000100:00100000:1.0:1789760752.352746:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730743317753 time 730782781004 state 0x2 ref 0 00000100:00100000:0.0:1789760752.367881:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760752.371934:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515719936 00000100:00100000:2.0:1789760752.371951:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515719936 00000100:00100000:3.0:1789760752.374787:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730743317753 time 730782781004 state 0x0 ref 1 00000100:00100000:3.0:1789760752.374806:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730783041420 time 730782781004 state 0x1 ref 1 00000100:00100000:3.0:1789760752.374950:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730783041420 time 730840237417 state 0x0 ref 1 00000100:00100000:3.0:1789760752.374974:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515719936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.375023:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23060480, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:3.0:1789760752.375032:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.375047:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760752.375052:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.375057:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.375070:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.449158:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972947 is committed 00002000:00010000:1.0:1789760752.449345:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23060480 -> 23064576 00010000:00010000:1.0:1789760752.449360:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.449373:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.449381:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.449388:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760752.449393:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.449402:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.449422:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.449435:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ec5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.449526:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515719936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74561us (81649us total) trans 4294972947 rc 0/0 00000100:00100000:1.0:1789760752.449537:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5659 00000100:00100000:1.0:1789760752.449547:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730783041420 time 730840237417 state 0x2 ref 0 00000100:00100000:0.0:1789760752.453971:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.454033:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720064 00000100:00100000:3.0:1789760752.454044:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720064 00000100:00100000:3.0:1789760752.454096:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730783041420 time 730840237417 state 0x0 ref 1 00000100:00100000:3.0:1789760752.454112:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730840497833 time 730840237417 state 0x1 ref 1 00000100:00100000:3.0:1789760752.454141:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730840497833 time 730919430140 state 0x0 ref 1 00000100:00100000:3.0:1789760752.454154:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515720064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.454200:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23064576, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:3.0:1789760752.454208:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.454224:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760752.454229:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.454235:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.454248:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.508790:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972948 is committed 00002000:00010000:1.0:1789760752.508997:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23064576 -> 23068672 00010000:00010000:1.0:1789760752.509013:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.509025:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.509034:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.509041:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760752.509045:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.509055:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.509080:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.509088:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ec63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.509253:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515720064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55096us (55286us total) trans 4294972948 rc 0/0 00000100:00100000:1.0:1789760752.509263:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5660 00000100:00100000:1.0:1789760752.509272:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730840497833 time 730919430140 state 0x2 ref 0 00000100:00100000:2.0:1789760752.526885:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.526987:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720192 00000100:00100000:3.0:1789760752.526997:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720192 00000100:00100000:3.0:1789760752.527050:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730840497833 time 730919430140 state 0x0 ref 1 00000100:00100000:3.0:1789760752.527065:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730919690556 time 730919430140 state 0x1 ref 1 00000100:00100000:3.0:1789760752.527098:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730919690556 time 730992386533 state 0x0 ref 1 00000100:00100000:3.0:1789760752.527112:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515720192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.527158:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23068672, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb007 00010000:00010000:3.0:1789760752.527167:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.527195:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760752.527200:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.527207:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.527222:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.561736:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.561772:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.561794:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.561865:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720448 00000100:00100000:3.0:1789760752.561874:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720448 00000100:00100000:3.0:1789760752.561943:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720576 00000100:00100000:3.0:1789760752.561946:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720576 00000100:00100000:3.0:1789760752.561970:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 466 00000100:00100000:3.0:1789760752.561980:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2363:x1876699515720448:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760752.562050:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2363:x1876699515720448:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 71us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760752.562057:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 466 00000100:00100000:3.0:1789760752.562075:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 467 00000100:00100000:3.0:1789760752.562080:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2363:x1876699515720576:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760752.562103:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2363:x1876699515720576:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 23us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760752.562110:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 467 00000100:00080000:1.0:1789760752.562262:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760752.563843:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720320 00000100:00100000:0.0:1789760752.563854:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720320 00000100:00100000:0.0:1789760752.563947:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 396 00000100:00100000:0.0:1789760752.563957:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909352450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2363:x1876699515720320:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760752.564023:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909352450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2363:x1876699515720320:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 67us (2249us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760752.564029:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 396 00000001:00080000:3.0:1789760752.569886:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972949 is committed 00002000:00010000:0.0:1789760752.570008:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23068672 -> 23072768 00002000:00010000:0.0:1789760752.570014:0:6712:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:0.0:1789760752.570026:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.570037:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.570046:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.570052:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.570056:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.570066:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.570087:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.570094:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ec6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.570194:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515720192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43080us (43374us total) trans 4294972949 rc 0/0 00000100:00100000:0.0:1789760752.570205:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5661 00000100:00100000:0.0:1789760752.570215:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730919690556 time 730992386533 state 0x2 ref 0 00000100:00100000:3.0:1789760752.579897:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760752.579979:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720704 00000100:00100000:0.0:1789760752.579989:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720704 00000100:00100000:0.0:1789760752.580043:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730919690556 time 730992386533 state 0x0 ref 1 00000100:00100000:0.0:1789760752.580059:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730992646949 time 730992386533 state 0x1 ref 1 00000100:00100000:0.0:1789760752.580091:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730992646949 time 731045380726 state 0x0 ref 1 00000100:00100000:0.0:1789760752.580104:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515720704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760752.580157:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23072768, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:0.0:1789760752.580167:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.580314:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.580318:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.580325:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.580340:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.625815:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972950 is committed 00002000:00010000:0.0:1789760752.626383:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23072768 -> 23076864 00010000:00010000:0.0:1789760752.626398:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.626411:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.626825:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.626834:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.626839:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.626849:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.626870:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.626878:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.627038:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515720704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46930us (47329us total) trans 4294972950 rc 0/0 00000100:00100000:0.0:1789760752.627049:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5662 00000100:00100000:0.0:1789760752.627058:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730992646949 time 731045380726 state 0x2 ref 0 00000100:00100000:3.0:1789760752.637805:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760752.637959:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720832 00000100:00100000:2.0:1789760752.637969:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720832 00000100:00100000:2.0:1789760752.638023:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 730992646949 time 731045380726 state 0x0 ref 1 00000100:00100000:2.0:1789760752.638038:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731045641142 time 731045380726 state 0x1 ref 1 00000100:00100000:2.0:1789760752.638069:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731045641142 time 731103357899 state 0x0 ref 1 00000100:00100000:2.0:1789760752.638081:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515720832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760752.638139:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23076864, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:2.0:1789760752.638149:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.638164:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.638169:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.638183:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.638210:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.675939:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972951 is committed 00002000:00010000:0.0:1789760752.676036:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23076864 -> 23080960 00010000:00010000:0.0:1789760752.676051:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.676063:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.676074:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.676081:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.676085:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.676094:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.676119:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.676128:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1ec78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.676222:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515720832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38138us (38422us total) trans 4294972951 rc 0/0 00000100:00100000:0.0:1789760752.676232:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5663 00000100:00100000:0.0:1789760752.676242:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731045641142 time 731103357899 state 0x2 ref 0 00000100:00100000:2.0:1789760752.681973:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.682154:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515720960 00000100:00100000:1.0:1789760752.682164:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515720960 00000100:00100000:1.0:1789760752.682216:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731045641142 time 731103357899 state 0x0 ref 1 00000100:00100000:1.0:1789760752.682231:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731103618315 time 731103357899 state 0x1 ref 1 00000100:00100000:1.0:1789760752.682298:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731103618315 time 731147586753 state 0x0 ref 1 00000100:00100000:1.0:1789760752.682312:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515720960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760752.682360:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23080960, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:1.0:1789760752.682370:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.682386:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.682390:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.682396:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.682421:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.724115:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972952 is committed 00002000:00010000:3.0:1789760752.724325:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23080960 -> 23085056 00010000:00010000:3.0:1789760752.724343:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.724355:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.724366:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.724374:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760752.724379:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.724389:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.724409:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.724418:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ec7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.724516:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515720960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42202us (42547us total) trans 4294972952 rc 0/0 00000100:00100000:3.0:1789760752.724526:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5664 00000100:00100000:3.0:1789760752.724535:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731103618315 time 731147586753 state 0x2 ref 0 00000100:00100000:3.0:1789760752.727804:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.727981:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721088 00000100:00100000:1.0:1789760752.727991:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721088 00000100:00100000:1.0:1789760752.728044:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731103618315 time 731147586753 state 0x0 ref 1 00000100:00100000:1.0:1789760752.728061:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731147847169 time 731147586753 state 0x1 ref 1 00000100:00100000:1.0:1789760752.728079:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731147847169 time 731193369795 state 0x0 ref 1 00000100:00100000:1.0:1789760752.728090:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760752.728136:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23085056, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:1.0:1789760752.728144:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.728158:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.728162:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.728168:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.728182:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.753455:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972953 is committed 00002000:00010000:0.0:1789760752.753743:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23085056 -> 23089152 00010000:00010000:0.0:1789760752.753758:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.753770:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.753779:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.753785:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.753789:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.753802:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.753833:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.753841:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ec86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.753936:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25842us (26137us total) trans 4294972953 rc 0/0 00000100:00100000:0.0:1789760752.753946:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5665 00000100:00100000:0.0:1789760752.753954:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731147847169 time 731193369795 state 0x2 ref 0 00000100:00100000:1.0:1789760752.757558:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760752.757669:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721216 00000100:00100000:3.0:1789760752.757679:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721216 00000100:00100000:3.0:1789760752.757758:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731147847169 time 731193369795 state 0x0 ref 1 00000100:00100000:3.0:1789760752.757773:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731193630211 time 731193369795 state 0x1 ref 1 00000100:00100000:3.0:1789760752.757798:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731193630211 time 731223087682 state 0x0 ref 1 00000100:00100000:3.0:1789760752.757810:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760752.757868:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23089152, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:3.0:1789760752.757878:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.757894:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760752.757910:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.757917:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.757937:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.784171:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23089152 -> 23093248 00010000:00010000:1.0:1789760752.784192:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.784204:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.784214:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.784221:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760752.784225:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.784234:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.784253:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.784260:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ec8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.784438:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972954 is committed 00000100:00100000:1.0:1789760752.785231:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27416us (27686us total) trans 4294972954 rc 0/0 00000100:00100000:1.0:1789760752.785244:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5666 00000100:00100000:1.0:1789760752.785254:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731193630211 time 731223087682 state 0x2 ref 0 00000100:00100000:2.0:1789760752.789053:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760752.789279:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721344 00000100:00100000:0.0:1789760752.789290:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721344 00000100:00100000:0.0:1789760752.789346:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731193630211 time 731223087682 state 0x0 ref 1 00000100:00100000:0.0:1789760752.789361:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731223348098 time 731223087682 state 0x1 ref 1 00000100:00100000:0.0:1789760752.789392:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731223348098 time 731254681477 state 0x0 ref 1 00000100:00100000:0.0:1789760752.789405:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515721344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760752.789470:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23093248, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:0.0:1789760752.789480:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.789498:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.789502:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.789508:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.789521:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.838246:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23093248 -> 23097344 00010000:00010000:0.0:1789760752.838266:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.838279:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.838289:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.838296:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.838301:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.838311:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.838337:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.838345:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ec94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.838437:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515721344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49029us (49387us total) trans 4294972955 rc 0/0 00000100:00100000:0.0:1789760752.838449:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5667 00000100:00100000:0.0:1789760752.838459:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731223348098 time 731254681477 state 0x2 ref 0 00000001:00080000:0.0:1789760752.838499:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972955 is committed 00000100:00100000:3.0:1789760752.842833:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760752.848139:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721472 00000100:00100000:0.0:1789760752.848154:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721472 00000100:00100000:2.0:1789760752.848290:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731223348098 time 731254681477 state 0x0 ref 1 00000100:00100000:2.0:1789760752.848306:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731254941893 time 731254681477 state 0x1 ref 1 00000100:00100000:2.0:1789760752.849955:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731254941893 time 731315235387 state 0x0 ref 1 00000100:00100000:2.0:1789760752.849980:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cab050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515721472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760752.850035:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23097344, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xb807 00010000:00010000:2.0:1789760752.850045:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.850061:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.850066:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.850072:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.850087:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.945366:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972956 is committed 00002000:00010000:0.0:1789760752.945540:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23097344 -> 23101440 00002000:00010000:0.0:1789760752.945545:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760752 -> 1789760753 00002000:00010000:0.0:1789760752.945547:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760752 -> 1789760753 00010000:00010000:0.0:1789760752.945560:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.945573:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.945582:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.945596:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.945601:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.945610:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.945629:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.945637:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ec9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.945767:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cab050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515721472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 95792us (102950us total) trans 4294972956 rc 0/0 00000100:00100000:0.0:1789760752.945777:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5668 00000100:00100000:0.0:1789760752.945787:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731254941893 time 731315235387 state 0x2 ref 0 00000100:00100000:1.0:1789760752.951221:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760752.951310:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721600 00000100:00100000:0.0:1789760752.951320:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721600 00000100:00100000:0.0:1789760752.951375:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731254941893 time 731315235387 state 0x0 ref 1 00000100:00100000:0.0:1789760752.951390:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731315495803 time 731315235387 state 0x1 ref 1 00000100:00100000:0.0:1789760752.951420:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731315495803 time 731416709410 state 0x0 ref 1 00000100:00100000:0.0:1789760752.951436:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760752.951486:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23101440, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:0.0:1789760752.951495:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.951510:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.951514:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.951521:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.951537:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.958222:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760752.988158:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972957 is committed 00002000:00010000:0.0:1789760752.988196:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23101440 -> 23105536 00010000:00010000:0.0:1789760752.988211:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.988223:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.988232:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.988240:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760752.988245:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.988255:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.988289:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.988298:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1eca2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.988389:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36951us (37173us total) trans 4294972957 rc 0/0 00000100:00100000:0.0:1789760752.988399:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5669 00000100:00100000:0.0:1789760752.988408:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731315495803 time 731416709410 state 0x2 ref 0 00000100:00100000:3.0:1789760752.994674:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760752.994761:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721728 00000100:00100000:1.0:1789760752.994771:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721728 00000100:00100000:1.0:1789760752.994826:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731315495803 time 731416709410 state 0x0 ref 1 00000100:00100000:1.0:1789760752.994842:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731416969826 time 731416709410 state 0x1 ref 1 00000100:00100000:1.0:1789760752.994871:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731416969826 time 731460159966 state 0x0 ref 1 00000100:00100000:1.0:1789760752.994884:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760752.994943:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23105536, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:1.0:1789760752.994952:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.994977:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.994981:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.994987:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.995002:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.023882:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972958 is committed 00002000:00010000:0.0:1789760753.028027:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23105536 -> 23109632 00010000:00010000:0.0:1789760753.028050:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.028063:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.028071:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.028077:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760753.028082:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.028091:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.028108:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.028114:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eca9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.028207:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515721728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33319us (33537us total) trans 4294972958 rc 0/0 00000100:00100000:0.0:1789760753.028219:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5670 00000100:00100000:0.0:1789760753.028228:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731416969826 time 731460159966 state 0x2 ref 0 00000100:00100000:3.0:1789760753.035635:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760753.035742:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721856 00000100:00100000:0.0:1789760753.035751:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721856 00000100:00100000:0.0:1789760753.035797:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731416969826 time 731460159966 state 0x0 ref 1 00000100:00100000:0.0:1789760753.035812:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731460420382 time 731460159966 state 0x1 ref 1 00000100:00100000:0.0:1789760753.035848:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731460420382 time 731501137372 state 0x0 ref 1 00000100:00100000:0.0:1789760753.035862:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515721856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760753.035936:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23109632, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:0.0:1789760753.035946:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.036177:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760753.036182:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.036189:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.036205:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.074431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972959 is committed 00002000:00010000:1.0:1789760753.074695:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23109632 -> 23113728 00010000:00010000:1.0:1789760753.074773:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.074787:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.074796:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.074803:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.074808:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.074817:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.075175:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.075185:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.075279:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515721856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39415us (39647us total) trans 4294972959 rc 0/0 00000100:00100000:1.0:1789760753.075289:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5671 00000100:00100000:1.0:1789760753.075299:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731460420382 time 731501137372 state 0x2 ref 0 00000100:00100000:1.0:1789760753.079588:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760753.080160:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515721984 00000100:00100000:3.0:1789760753.080171:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515721984 00000100:00100000:3.0:1789760753.080398:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731460420382 time 731501137372 state 0x0 ref 1 00000100:00100000:3.0:1789760753.080413:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731501397788 time 731501137372 state 0x1 ref 1 00000100:00100000:3.0:1789760753.080439:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731501397788 time 731545728143 state 0x0 ref 1 00000100:00100000:3.0:1789760753.080452:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515721984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760753.080546:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23113728, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:3.0:1789760753.080556:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.080572:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760753.080578:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.080584:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.080604:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.116155:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23113728 -> 23117824 00010000:00010000:1.0:1789760753.116176:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116187:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116196:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116213:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.116219:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116229:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116248:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116262:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ecb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.116354:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515721984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35899us (36769us total) trans 4294972960 rc 0/0 00000100:00100000:1.0:1789760753.116365:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5672 00000100:00100000:1.0:1789760753.116374:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731501397788 time 731545728143 state 0x2 ref 0 00000001:00080000:2.0:1789760753.117054:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972960 is committed 00000100:00100000:3.0:1789760753.121449:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760753.121695:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722112 00000100:00100000:0.0:1789760753.121727:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722112 00000100:00100000:0.0:1789760753.121780:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731501397788 time 731545728143 state 0x0 ref 1 00000100:00100000:0.0:1789760753.121796:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731545988559 time 731545728143 state 0x1 ref 1 00000100:00100000:2.0:1789760753.121935:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731545988559 time 731587222528 state 0x0 ref 1 00000100:00100000:2.0:1789760753.121951:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.121997:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23117824, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:2.0:1789760753.122006:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.122022:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760753.122026:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.122032:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.122046:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.155478:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972961 is committed 00002000:00010000:3.0:1789760753.155687:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23117824 -> 23121920 00010000:00010000:3.0:1789760753.155702:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.155776:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.155785:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.155792:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760753.155797:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.155807:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.155878:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.155888:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ecbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.156074:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34122us (34629us total) trans 4294972961 rc 0/0 00000100:00100000:3.0:1789760753.156285:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5673 00000100:00100000:3.0:1789760753.156296:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731545988559 time 731587222528 state 0x2 ref 0 00000100:00100000:3.0:1789760753.162971:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760753.163320:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722240 00000100:00100000:2.0:1789760753.163331:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722240 00000100:00100000:2.0:1789760753.163383:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731545988559 time 731587222528 state 0x0 ref 1 00000100:00100000:2.0:1789760753.163398:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731587482944 time 731587222528 state 0x1 ref 1 00000100:00100000:2.0:1789760753.163429:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731587482944 time 731628718555 state 0x0 ref 1 00000100:00100000:2.0:1789760753.163442:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515722240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.163487:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23121920, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:2.0:1789760753.163495:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.163510:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760753.163514:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.163521:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.163535:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.199376:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23121920 -> 23126016 00010000:00010000:3.0:1789760753.199395:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.199408:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.199417:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.199424:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760753.199429:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.199438:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.199455:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.199469:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ecc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.199558:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515722240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36114us (36591us total) trans 4294972962 rc 0/0 00000100:00100000:3.0:1789760753.199568:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5674 00000100:00100000:3.0:1789760753.199577:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731587482944 time 731628718555 state 0x2 ref 0 00000001:00080000:1.0:1789760753.199651:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972962 is committed 00000100:00100000:1.0:1789760753.205056:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760753.205185:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722368 00000100:00100000:0.0:1789760753.205196:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722368 00000100:00100000:0.0:1789760753.205310:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731587482944 time 731628718555 state 0x0 ref 1 00000100:00100000:0.0:1789760753.205325:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731628978971 time 731628718555 state 0x1 ref 1 00000100:00100000:0.0:1789760753.205355:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731628978971 time 731670644018 state 0x0 ref 1 00000100:00100000:0.0:1789760753.205366:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760753.205409:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23126016, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:0.0:1789760753.205420:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.205434:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760753.205437:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.205442:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.205452:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.248369:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972963 is committed 00002000:00010000:0.0:1789760753.249138:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23126016 -> 23130112 00010000:00010000:0.0:1789760753.249154:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.249167:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.249176:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.249183:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760753.249187:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.249196:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.249288:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.249299:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eccc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.249509:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44140us (44458us total) trans 4294972963 rc 0/0 00000100:00100000:0.0:1789760753.249521:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5675 00000100:00100000:0.0:1789760753.249530:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731628978971 time 731670644018 state 0x2 ref 0 00000100:00100000:0.0:1789760753.258018:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760753.258197:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722496 00000100:00100000:1.0:1789760753.258215:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722496 00000100:00100000:1.0:1789760753.258266:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731628978971 time 731670644018 state 0x0 ref 1 00000100:00100000:1.0:1789760753.258282:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731670904434 time 731670644018 state 0x1 ref 1 00000100:00100000:1.0:1789760753.258313:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731670904434 time 731723602620 state 0x0 ref 1 00000100:00100000:1.0:1789760753.258326:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515722496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760753.258373:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23130112, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:1.0:1789760753.258392:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.258408:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760753.258413:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.258420:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.258433:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.317561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972964 is committed 00002000:00010000:1.0:1789760753.317660:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23130112 -> 23134208 00010000:00010000:1.0:1789760753.317675:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.317688:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.317698:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.317733:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.317739:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.317748:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.317770:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.317777:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ecd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.318017:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515722496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59689us (60003us total) trans 4294972964 rc 0/0 00000100:00100000:1.0:1789760753.318028:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5676 00000100:00100000:1.0:1789760753.318037:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731670904434 time 731723602620 state 0x2 ref 0 00000100:00100000:3.0:1789760753.322094:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760753.322166:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722624 00000100:00100000:1.0:1789760753.322175:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722624 00000100:00100000:1.0:1789760753.322223:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731670904434 time 731723602620 state 0x0 ref 1 00000100:00100000:1.0:1789760753.322236:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731723863036 time 731723602620 state 0x1 ref 1 00000100:00100000:1.0:1789760753.322263:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731723863036 time 731787552354 state 0x0 ref 1 00000100:00100000:1.0:1789760753.322279:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760753.322330:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23134208, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:1.0:1789760753.322339:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.322355:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760753.322359:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.322366:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.322379:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.377655:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23134208 -> 23138304 00010000:00010000:1.0:1789760753.377675:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.377687:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.377696:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.377724:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.377729:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.377737:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.377755:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.377762:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ecda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.378018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972965 is committed 00000100:00100000:1.0:1789760753.378197:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55914us (56107us total) trans 4294972965 rc 0/0 00000100:00100000:1.0:1789760753.378212:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5677 00000100:00100000:1.0:1789760753.378222:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731723863036 time 731787552354 state 0x2 ref 0 00000100:00100000:1.0:1789760753.383399:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760753.383533:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722752 00000100:00100000:3.0:1789760753.383544:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722752 00000100:00100000:3.0:1789760753.383598:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731723863036 time 731787552354 state 0x0 ref 1 00000100:00100000:3.0:1789760753.383615:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731787812770 time 731787552354 state 0x1 ref 1 00000100:00100000:3.0:1789760753.383683:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731787812770 time 731848970880 state 0x0 ref 1 00000100:00100000:3.0:1789760753.383696:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760753.383765:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23138304, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:3.0:1789760753.383775:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.383792:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760753.383797:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.383804:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.383827:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.431828:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972966 is committed 00002000:00010000:1.0:1789760753.434034:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23138304 -> 23142400 00010000:00010000:1.0:1789760753.434053:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434068:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434077:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434085:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.434090:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434099:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434119:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434126:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ece1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.434225:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515722752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50527us (50830us total) trans 4294972966 rc 0/0 00000100:00100000:1.0:1789760753.434424:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5678 00000100:00100000:1.0:1789760753.434437:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731787812770 time 731848970880 state 0x2 ref 0 00000100:00100000:0.0:1789760753.438858:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760753.440060:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515722880 00000100:00100000:3.0:1789760753.440074:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515722880 00000100:00100000:3.0:1789760753.440131:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731787812770 time 731848970880 state 0x0 ref 1 00000100:00100000:3.0:1789760753.440146:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731849231296 time 731848970880 state 0x1 ref 1 00000100:00100000:3.0:1789760753.440183:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731849231296 time 731905471888 state 0x0 ref 1 00000100:00100000:3.0:1789760753.440196:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515722880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760753.440408:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23142400, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:3.0:1789760753.440418:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.440435:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760753.440439:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.440445:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.440459:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.497660:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972967 is committed 00002000:00010000:3.0:1789760753.498041:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23142400 -> 23146496 00010000:00010000:3.0:1789760753.498056:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.498070:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.498081:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.498088:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760753.498093:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.498102:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.498121:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.498129:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ece8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.498228:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515722880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58031us (59527us total) trans 4294972967 rc 0/0 00000100:00100000:3.0:1789760753.498250:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5679 00000100:00100000:3.0:1789760753.498260:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731849231296 time 731905471888 state 0x2 ref 0 00000100:00100000:3.0:1789760753.502437:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760753.503588:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723008 00000100:00100000:2.0:1789760753.503601:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723008 00000100:00100000:2.0:1789760753.503655:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731849231296 time 731905471888 state 0x0 ref 1 00000100:00100000:2.0:1789760753.503670:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731905732304 time 731905471888 state 0x1 ref 1 00000100:00100000:2.0:1789760753.503737:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731905732304 time 731968994970 state 0x0 ref 1 00000100:00100000:2.0:1789760753.503752:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515723008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.503797:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23146496, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:2.0:1789760753.503807:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.503823:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760753.503827:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.503834:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.503847:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.553329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972968 is committed 00002000:00010000:0.0:1789760753.553440:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23146496 -> 23150592 00010000:00010000:0.0:1789760753.553457:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.553469:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.553477:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.553484:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760753.553490:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.553498:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.553519:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.553526:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ecef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.553620:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515723008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49867us (51189us total) trans 4294972968 rc 0/0 00000100:00100000:0.0:1789760753.553630:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5680 00000100:00100000:0.0:1789760753.553639:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731905732304 time 731968994970 state 0x2 ref 0 00000100:00100000:0.0:1789760753.560611:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760753.560737:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723136 00000100:00100000:2.0:1789760753.560748:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723136 00000100:00100000:2.0:1789760753.560802:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731905732304 time 731968994970 state 0x0 ref 1 00000100:00100000:2.0:1789760753.560931:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731969255386 time 731968994970 state 0x1 ref 1 00000100:00100000:2.0:1789760753.560967:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731969255386 time 732026256266 state 0x0 ref 1 00000100:00100000:2.0:1789760753.560980:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515723136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.561097:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23150592, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:2.0:1789760753.561106:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.561123:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.561128:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.561134:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.561148:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.596272:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972969 is committed 00002000:00010000:2.0:1789760753.596401:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23150592 -> 23154688 00010000:00010000:2.0:1789760753.596416:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.596429:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.596438:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.596445:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760753.596450:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.596459:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.596477:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.596485:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ecf6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.596654:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515723136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35670us (36048us total) trans 4294972969 rc 0/0 00000100:00100000:2.0:1789760753.596669:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5681 00000100:00100000:2.0:1789760753.596679:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731969255386 time 732026256266 state 0x2 ref 0 00000100:00100000:2.0:1789760753.601618:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760753.601730:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723264 00000100:00100000:0.0:1789760753.601740:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723264 00000100:00100000:0.0:1789760753.601788:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 731969255386 time 732026256266 state 0x0 ref 1 00000100:00100000:0.0:1789760753.601803:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732026516682 time 732026256266 state 0x1 ref 1 00000100:00100000:0.0:1789760753.601836:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732026516682 time 732067125143 state 0x0 ref 1 00000100:00100000:0.0:1789760753.601847:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515723264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760753.601901:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23154688, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:0.0:1789760753.601911:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.601927:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760753.601931:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.601938:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.601952:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.641264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972970 is committed 00002000:00010000:1.0:1789760753.641516:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23154688 -> 23158784 00010000:00010000:1.0:1789760753.641532:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.641615:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.641625:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.641632:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.641637:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.641646:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.641665:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.641678:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ecfd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.641853:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515723264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40002us (40239us total) trans 4294972970 rc 0/0 00000100:00100000:1.0:1789760753.641864:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5682 00000100:00100000:1.0:1789760753.641874:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732026516682 time 732067125143 state 0x2 ref 0 00000100:00100000:0.0:1789760753.646901:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760753.646973:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723392 00000100:00100000:1.0:1789760753.646983:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723392 00000100:00100000:1.0:1789760753.647044:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732026516682 time 732067125143 state 0x0 ref 1 00000100:00100000:1.0:1789760753.647060:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732067385559 time 732067125143 state 0x1 ref 1 00000100:00100000:1.0:1789760753.647085:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732067385559 time 732112374490 state 0x0 ref 1 00000100:00100000:1.0:1789760753.647095:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515723392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760753.647140:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23158784, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:1.0:1789760753.647148:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.647163:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.647167:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.647173:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.647191:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.681412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972971 is committed 00002000:00010000:1.0:1789760753.681481:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23158784 -> 23162880 00010000:00010000:1.0:1789760753.681497:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.681509:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.681517:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.681524:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760753.681528:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.681776:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.681797:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.681817:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ed04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.681904:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515723392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34796us (35010us total) trans 4294972971 rc 0/0 00000100:00100000:1.0:1789760753.681915:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5683 00000100:00100000:1.0:1789760753.681924:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732067385559 time 732112374490 state 0x2 ref 0 00000100:00100000:0.0:1789760753.686448:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760753.686792:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723520 00000100:00100000:1.0:1789760753.686802:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723520 00000100:00100000:1.0:1789760753.686863:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732067385559 time 732112374490 state 0x0 ref 1 00000100:00100000:1.0:1789760753.686879:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732112634906 time 732112374490 state 0x1 ref 1 00000100:00100000:1.0:1789760753.686916:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732112634906 time 732152204894 state 0x0 ref 1 00000100:00100000:1.0:1789760753.686928:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515723520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760753.686974:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23162880, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:1.0:1789760753.686984:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.687000:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.687005:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.687011:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.687026:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.717882:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972972 is committed 00002000:00010000:2.0:1789760753.718055:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23162880 -> 23166976 00010000:00010000:2.0:1789760753.718071:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.718085:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.718094:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.718103:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760753.718108:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.718117:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.718136:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.718143:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.718332:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515723520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31400us (31886us total) trans 4294972972 rc 0/0 00000100:00100000:2.0:1789760753.718343:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5684 00000100:00100000:2.0:1789760753.718352:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732112634906 time 732152204894 state 0x2 ref 0 00000100:00100000:1.0:1789760753.729465:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760753.732452:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723648 00000100:00100000:3.0:1789760753.732468:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723648 00000100:00100000:3.0:1789760753.732530:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732112634906 time 732152204894 state 0x0 ref 1 00000100:00100000:3.0:1789760753.732547:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732152465310 time 732152204894 state 0x1 ref 1 00000100:00100000:3.0:1789760753.732634:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732152465310 time 732197922880 state 0x0 ref 1 00000100:00100000:3.0:1789760753.732649:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515723648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760753.732698:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23166976, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:3.0:1789760753.732722:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732739:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.732743:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.732750:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.732764:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.782510:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972973 is committed 00002000:00010000:3.0:1789760753.782590:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23166976 -> 23171072 00010000:00010000:3.0:1789760753.782605:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.782619:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.782628:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.782635:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760753.782640:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.782650:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.782670:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.782677:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.782796:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515723648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50146us (53335us total) trans 4294972973 rc 0/0 00000100:00100000:3.0:1789760753.782807:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5685 00000100:00100000:3.0:1789760753.782829:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732152465310 time 732197922880 state 0x2 ref 0 00000100:00100000:0.0:1789760753.792411:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760753.792493:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723776 00000100:00100000:2.0:1789760753.792503:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723776 00000100:00100000:2.0:1789760753.792556:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732152465310 time 732197922880 state 0x0 ref 1 00000100:00100000:2.0:1789760753.792572:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732198183296 time 732197922880 state 0x1 ref 1 00000100:00100000:2.0:1789760753.792605:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732198183296 time 732257894165 state 0x0 ref 1 00000100:00100000:2.0:1789760753.792618:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515723776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.792671:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23171072, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:2.0:1789760753.792680:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.792698:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.792703:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.792745:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.792763:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.875732:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23171072 -> 23175168 00010000:00010000:2.0:1789760753.875753:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.875766:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.875776:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.875782:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760753.875787:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.875794:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.875821:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.875830:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.875932:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515723776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 83311us (83527us total) trans 4294972974 rc 0/0 00000100:00100000:2.0:1789760753.875945:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5686 00000100:00100000:2.0:1789760753.875955:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732198183296 time 732257894165 state 0x2 ref 0 00000001:00080000:0.0:1789760753.876887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972974 is committed 00000100:00100000:1.0:1789760753.883048:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760753.883506:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515723904 00000100:00100000:2.0:1789760753.883517:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515723904 00000100:00100000:2.0:1789760753.883573:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732198183296 time 732257894165 state 0x0 ref 1 00000100:00100000:2.0:1789760753.883589:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732258154581 time 732257894165 state 0x1 ref 1 00000100:00100000:2.0:1789760753.883663:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732258154581 time 732348951105 state 0x0 ref 1 00000100:00100000:2.0:1789760753.883677:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515723904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.883759:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23175168, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb807 00010000:00010000:2.0:1789760753.883769:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.883786:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.883791:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.883799:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.883814:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.924376:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972975 is committed 00002000:00010000:3.0:1789760753.924486:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23175168 -> 23179264 00002000:00010000:3.0:1789760753.924491:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:3.0:1789760753.924494:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00010000:00010000:3.0:1789760753.924507:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.924520:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.924530:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.924537:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760753.924542:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.924551:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.924572:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.924580:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.924677:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515723904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40998us (41634us total) trans 4294972975 rc 0/0 00000100:00100000:3.0:1789760753.924688:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5687 00000100:00100000:3.0:1789760753.924698:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732258154581 time 732348951105 state 0x2 ref 0 00000100:00100000:3.0:1789760753.930904:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760753.930975:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724032 00000100:00100000:1.0:1789760753.930985:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724032 00000100:00100000:1.0:1789760753.931039:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732258154581 time 732348951105 state 0x0 ref 1 00000100:00100000:1.0:1789760753.931054:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732349211521 time 732348951105 state 0x1 ref 1 00000100:00100000:1.0:1789760753.931087:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732349211521 time 732396375838 state 0x0 ref 1 00000100:00100000:1.0:1789760753.931101:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760753.931147:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23179264, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:1.0:1789760753.931156:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.931171:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.931175:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.931181:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.931195:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.964663:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972976 is committed 00002000:00010000:2.0:1789760753.964773:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23179264 -> 23183360 00010000:00010000:2.0:1789760753.964790:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.964804:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.964825:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.964832:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760753.964837:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.964846:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.964865:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.964873:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ed27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.964981:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33879us (34091us total) trans 4294972976 rc 0/0 00000100:00100000:2.0:1789760753.965765:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5688 00000100:00100000:2.0:1789760753.965781:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732349211521 time 732396375838 state 0x2 ref 0 00000100:00100000:1.0:1789760753.968472:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760753.968562:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724160 00000100:00100000:2.0:1789760753.968572:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724160 00000100:00100000:2.0:1789760753.968624:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732349211521 time 732396375838 state 0x0 ref 1 00000100:00100000:2.0:1789760753.968639:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732396636254 time 732396375838 state 0x1 ref 1 00000100:00100000:2.0:1789760753.968672:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732396636254 time 732433961147 state 0x0 ref 1 00000100:00100000:2.0:1789760753.968684:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760753.968758:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23183360, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760753.968768:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.968783:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.968789:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.968795:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.968932:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.983462:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760754.008256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972977 is committed 00002000:00010000:3.0:1789760754.008448:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23183360 -> 23187456 00010000:00010000:3.0:1789760754.008464:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.008477:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.008486:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.008493:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.008499:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.008509:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.008526:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.008535:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.008630:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39943us (40160us total) trans 4294972977 rc 0/0 00000100:00100000:3.0:1789760754.008642:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5689 00000100:00100000:3.0:1789760754.008653:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732396636254 time 732433961147 state 0x2 ref 0 00000100:00100000:1.0:1789760754.012490:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.012742:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724288 00000100:00100000:2.0:1789760754.012752:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724288 00000100:00100000:2.0:1789760754.012825:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732396636254 time 732433961147 state 0x0 ref 1 00000100:00100000:2.0:1789760754.012848:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732434221563 time 732433961147 state 0x1 ref 1 00000100:00100000:2.0:1789760754.012872:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732434221563 time 732478161799 state 0x0 ref 1 00000100:00100000:2.0:1789760754.012884:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515724288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.012943:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23187456, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760754.012951:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.012965:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760754.012969:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.012975:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.012989:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760754.013564:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760754.047264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972978 is committed 00002000:00010000:2.0:1789760754.047471:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23187456 -> 23191552 00010000:00010000:2.0:1789760754.047486:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.047499:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.047507:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.047514:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.047520:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.047528:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.047661:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.047669:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.047954:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515724288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35066us (35467us total) trans 4294972978 rc 0/0 00000100:00100000:2.0:1789760754.047966:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5690 00000100:00100000:2.0:1789760754.047979:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732434221563 time 732478161799 state 0x2 ref 0 00000100:00100000:1.0:1789760754.051637:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.051779:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724416 00000100:00100000:3.0:1789760754.051789:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724416 00000100:00100000:3.0:1789760754.051847:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732434221563 time 732478161799 state 0x0 ref 1 00000100:00100000:3.0:1789760754.051864:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732478422215 time 732478161799 state 0x1 ref 1 00000100:00100000:3.0:1789760754.051896:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732478422215 time 732517177470 state 0x0 ref 1 00000100:00100000:3.0:1789760754.051915:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.051964:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23191552, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.051973:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.051991:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.051995:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.052000:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.052014:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.079561:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972979 is committed 00002000:00010000:3.0:1789760754.079659:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23191552 -> 23195648 00010000:00010000:3.0:1789760754.079675:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.079687:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.079728:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.079737:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.079743:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.079751:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.079770:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.079778:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.081956:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28077us (28364us total) trans 4294972979 rc 0/0 00000100:00100000:3.0:1789760754.081972:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5691 00000100:00100000:3.0:1789760754.081983:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732478422215 time 732517177470 state 0x2 ref 0 00000100:00100000:0.0:1789760754.085592:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.085852:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724544 00000100:00100000:3.0:1789760754.085863:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724544 00000100:00100000:3.0:1789760754.085925:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732478422215 time 732517177470 state 0x0 ref 1 00000100:00100000:3.0:1789760754.085941:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732517437886 time 732517177470 state 0x1 ref 1 00000100:00100000:3.0:1789760754.085964:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732517437886 time 732551253305 state 0x0 ref 1 00000100:00100000:3.0:1789760754.085976:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515724544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.086034:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23195648, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.086043:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.086060:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.086065:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.086070:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.086084:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.114401:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972980 is committed 00002000:00010000:0.0:1789760754.114464:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23195648 -> 23199744 00010000:00010000:0.0:1789760754.114478:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.114491:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.114505:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.114514:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760754.114519:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.114528:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.114549:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.114557:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ed43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.114657:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515724544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28677us (29070us total) trans 4294972980 rc 0/0 00000100:00100000:0.0:1789760754.114667:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5692 00000100:00100000:0.0:1789760754.114677:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732517437886 time 732551253305 state 0x2 ref 0 00000100:00100000:0.0:1789760754.118978:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.119502:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724672 00000100:00100000:3.0:1789760754.119513:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724672 00000100:00100000:3.0:1789760754.119634:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732517437886 time 732551253305 state 0x0 ref 1 00000100:00100000:3.0:1789760754.119651:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732551513721 time 732551253305 state 0x1 ref 1 00000100:00100000:3.0:1789760754.119741:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732551513721 time 732585029016 state 0x0 ref 1 00000100:00100000:3.0:1789760754.119754:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.119802:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23199744, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.119822:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.119839:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.119843:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.119850:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.119864:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.154671:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972981 is committed 00002000:00010000:0.0:1789760754.154986:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23199744 -> 23203840 00010000:00010000:0.0:1789760754.155003:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.155017:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.155025:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.155032:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760754.155037:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.155046:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.155067:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.155075:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.155178:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515724672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35420us (36204us total) trans 4294972981 rc 0/0 00000100:00100000:0.0:1789760754.155190:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5693 00000100:00100000:0.0:1789760754.155201:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732551513721 time 732585029016 state 0x2 ref 0 00000100:00100000:0.0:1789760754.168393:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.168509:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724800 00000100:00100000:2.0:1789760754.168519:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724800 00000100:00100000:2.0:1789760754.168574:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732551513721 time 732585029016 state 0x0 ref 1 00000100:00100000:2.0:1789760754.168589:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732585289432 time 732585029016 state 0x1 ref 1 00000100:00100000:2.0:1789760754.168621:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732585289432 time 732633909720 state 0x0 ref 1 00000100:00100000:2.0:1789760754.168633:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515724800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.168682:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23203840, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760754.168692:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.168836:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760754.168841:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.168848:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.168862:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.175030:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760754.175052:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544414720:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760754.175119:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760754.175272:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760754.175282:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544414848:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760754.175314:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760754.175348:0:11703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414720 00000100:00100000:2.0:1789760754.175355:0:11703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414720 00000100:00100000:2.0:1789760754.175403:0:11703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414848 00000100:00100000:2.0:1789760754.175406:0:11703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414848 00000100:00100000:2.0:1789760754.175430:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789760754.175439:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544414720:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760754.175462:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3834642432 avail 3818913792 00000100:00100000:2.0:1789760754.175505:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544414720:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760754.175512:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789760754.175529:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1789760754.175535:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544414848:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760754.175557:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760754.175585:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544414848:12345-0@lo:13:osp-pre-1-0.0 Request processed in 51us (271us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760754.175590:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00080000:2.0:1789760754.175604:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760754.175607:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760754.175641:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544414848:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760754.175671:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544414720:0@lo:13:osp-pre-0-0.0 00002000:00010000:3.0:1789760754.218595:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23203840 -> 23207936 00010000:00010000:3.0:1789760754.218615:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.218628:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.218636:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.218643:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.218648:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.218657:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.218675:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.218683:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1ed51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.218802:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515724800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50166us (50414us total) trans 4294972982 rc 0/0 00000100:00100000:3.0:1789760754.218823:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5694 00000100:00100000:3.0:1789760754.218833:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732585289432 time 732633909720 state 0x2 ref 0 00000001:00080000:0.0:1789760754.219306:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972982 is committed 00000100:00100000:1.0:1789760754.223093:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760754.223175:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515724928 00000100:00100000:0.0:1789760754.223185:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515724928 00000100:00100000:0.0:1789760754.223237:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732585289432 time 732633909720 state 0x0 ref 1 00000100:00100000:0.0:1789760754.223251:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732634170136 time 732633909720 state 0x1 ref 1 00000100:00100000:0.0:1789760754.223281:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732634170136 time 732688570750 state 0x0 ref 1 00000100:00100000:0.0:1789760754.223294:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515724928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760754.223343:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23207936, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:0.0:1789760754.223352:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.223367:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760754.223371:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.223377:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.223389:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760754.254643:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23207936 -> 23212032 00010000:00010000:1.0:1789760754.254662:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.254674:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.254682:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.254688:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760754.254693:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.254700:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.254752:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.254758:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ed58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.254853:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515724928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31556us (31764us total) trans 4294972983 rc 0/0 00000100:00100000:1.0:1789760754.254866:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5695 00000100:00100000:1.0:1789760754.254875:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732634170136 time 732688570750 state 0x2 ref 0 00000001:00080000:0.0:1789760754.255421:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972983 is committed 00000100:00100000:0.0:1789760754.267187:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.267283:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725056 00000100:00100000:2.0:1789760754.267293:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725056 00000100:00100000:2.0:1789760754.267347:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732634170136 time 732688570750 state 0x0 ref 1 00000100:00100000:2.0:1789760754.267363:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732688831166 time 732688570750 state 0x1 ref 1 00000100:00100000:2.0:1789760754.267427:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732688831166 time 732732715415 state 0x0 ref 1 00000100:00100000:2.0:1789760754.267439:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515725056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.267485:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23212032, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760754.267495:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.267511:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760754.267516:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.267522:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.267535:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.290128:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23212032 -> 23216128 00010000:00010000:3.0:1789760754.290155:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.290157:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972984 is committed 00010000:00010000:3.0:1789760754.290168:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.290177:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.290185:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.290190:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.290200:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.290219:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.290226:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ed5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.290318:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515725056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22877us (23136us total) trans 4294972984 rc 0/0 00000100:00100000:3.0:1789760754.290328:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5696 00000100:00100000:3.0:1789760754.290339:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732688831166 time 732732715415 state 0x2 ref 0 00000100:00100000:2.0:1789760754.297699:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.298366:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725184 00000100:00100000:3.0:1789760754.298376:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725184 00000100:00100000:3.0:1789760754.298438:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732688831166 time 732732715415 state 0x0 ref 1 00000100:00100000:3.0:1789760754.298453:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732732975831 time 732732715415 state 0x1 ref 1 00000100:00100000:3.0:1789760754.298484:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732732975831 time 732763773461 state 0x0 ref 1 00000100:00100000:3.0:1789760754.298496:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515725184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.298540:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23216128, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.298548:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.298562:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.298566:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.298572:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.298585:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.331557:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23216128 -> 23220224 00010000:00010000:3.0:1789760754.331576:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.331597:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.331606:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.331613:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.331618:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.331628:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.331647:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.331654:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.332152:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515725184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33290us (34094us total) trans 4294972985 rc 0/0 00000100:00100000:3.0:1789760754.332163:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5697 00000100:00100000:3.0:1789760754.332173:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732732975831 time 732763773461 state 0x2 ref 0 00000001:00080000:0.0:1789760754.332344:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972985 is committed 00000100:00100000:1.0:1789760754.348389:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.348491:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725312 00000100:00100000:3.0:1789760754.348503:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725312 00000100:00100000:3.0:1789760754.348559:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732732975831 time 732763773461 state 0x0 ref 1 00000100:00100000:3.0:1789760754.348575:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732764033877 time 732763773461 state 0x1 ref 1 00000100:00100000:3.0:1789760754.348609:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732764033877 time 732813897612 state 0x0 ref 1 00000100:00100000:3.0:1789760754.348622:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515725312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.348672:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23220224, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.348681:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.348697:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.348702:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.348724:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.348740:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.412505:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23220224 -> 23224320 00010000:00010000:3.0:1789760754.412526:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.412540:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.412555:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.412563:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.412568:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.412577:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.412598:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.412607:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.412702:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515725312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64077us (64317us total) trans 4294972986 rc 0/0 00000100:00100000:3.0:1789760754.412741:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5698 00000100:00100000:3.0:1789760754.412752:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732764033877 time 732813897612 state 0x2 ref 0 00000001:00080000:1.0:1789760754.414303:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972986 is committed 00000100:00100000:3.0:1789760754.417448:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760754.417545:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725440 00000100:00100000:0.0:1789760754.417556:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725440 00000100:00100000:0.0:1789760754.417607:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732764033877 time 732813897612 state 0x0 ref 1 00000100:00100000:0.0:1789760754.417622:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732814158028 time 732813897612 state 0x1 ref 1 00000100:00100000:0.0:1789760754.417651:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732814158028 time 732882940144 state 0x0 ref 1 00000100:00100000:0.0:1789760754.417662:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515725440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760754.417739:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23224320, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:0.0:1789760754.417748:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.417764:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760754.417769:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.417776:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.417792:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.457568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972987 is committed 00002000:00010000:3.0:1789760754.457810:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23224320 -> 23228416 00010000:00010000:3.0:1789760754.457825:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.457842:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.457851:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.457859:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.457864:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.457873:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.457903:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.457912:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ed74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.458021:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515725440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40354us (40576us total) trans 4294972987 rc 0/0 00000100:00100000:3.0:1789760754.458032:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5699 00000100:00100000:3.0:1789760754.458043:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732814158028 time 732882940144 state 0x2 ref 0 00000100:00100000:0.0:1789760754.463919:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.463982:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725568 00000100:00100000:2.0:1789760754.463992:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725568 00000100:00100000:2.0:1789760754.464045:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732814158028 time 732882940144 state 0x0 ref 1 00000100:00100000:2.0:1789760754.464058:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732883200560 time 732882940144 state 0x1 ref 1 00000100:00100000:2.0:1789760754.464082:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732883200560 time 732929372808 state 0x0 ref 1 00000100:00100000:2.0:1789760754.464091:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515725568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.464132:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23228416, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760754.464139:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.464152:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760754.464156:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.464160:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.464171:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.493481:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972988 is committed 00002000:00010000:2.0:1789760754.493576:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23228416 -> 23232512 00010000:00010000:2.0:1789760754.493592:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.493611:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.493620:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.493627:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.493632:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.493642:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.493662:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.493669:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1ed7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.494539:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515725568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29709us (29891us total) trans 4294972988 rc 0/0 00000100:00100000:2.0:1789760754.494556:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5700 00000100:00100000:2.0:1789760754.494566:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732883200560 time 732929372808 state 0x2 ref 0 00000100:00100000:1.0:1789760754.498213:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.498436:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725696 00000100:00100000:3.0:1789760754.498446:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725696 00000100:00100000:3.0:1789760754.498498:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732883200560 time 732929372808 state 0x0 ref 1 00000100:00100000:3.0:1789760754.498515:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732929633224 time 732929372808 state 0x1 ref 1 00000100:00100000:3.0:1789760754.498547:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732929633224 time 732963835844 state 0x0 ref 1 00000100:00100000:3.0:1789760754.498560:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515725696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.498611:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23232512, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.498621:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.498637:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.498641:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.498647:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.498662:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.524213:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972989 is committed 00002000:00010000:3.0:1789760754.524314:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23232512 -> 23236608 00010000:00010000:3.0:1789760754.524331:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.524343:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.524356:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.524364:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.524369:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.524377:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.524396:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.524404:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.524498:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515725696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25937us (26291us total) trans 4294972989 rc 0/0 00000100:00100000:3.0:1789760754.524509:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5701 00000100:00100000:3.0:1789760754.524518:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732929633224 time 732963835844 state 0x2 ref 0 00000100:00100000:1.0:1789760754.527480:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.527672:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725824 00000100:00100000:3.0:1789760754.527683:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725824 00000100:00100000:3.0:1789760754.527878:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732929633224 time 732963835844 state 0x0 ref 1 00000100:00100000:3.0:1789760754.527904:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732964096260 time 732963835844 state 0x1 ref 1 00000100:00100000:3.0:1789760754.527975:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732964096260 time 732993263919 state 0x0 ref 1 00000100:00100000:3.0:1789760754.527988:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515725824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.528037:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23236608, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.528046:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.528062:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760754.528067:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.528073:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.528087:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.558746:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23236608 -> 23240704 00010000:00010000:2.0:1789760754.558768:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.558781:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.558789:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.558796:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.558800:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.558821:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.558843:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.558851:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1ed89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.558954:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515725824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30962us (31478us total) trans 4294972990 rc 0/0 00000100:00100000:2.0:1789760754.558964:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5702 00000100:00100000:2.0:1789760754.558974:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732964096260 time 732993263919 state 0x2 ref 0 00000001:00080000:0.0:1789760754.560993:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972990 is committed 00000100:00100000:0.0:1789760754.562811:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.562909:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515725952 00000100:00100000:3.0:1789760754.562919:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515725952 00000100:00100000:3.0:1789760754.562973:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732964096260 time 732993263919 state 0x0 ref 1 00000100:00100000:3.0:1789760754.562988:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732993524335 time 732993263919 state 0x1 ref 1 00000100:00100000:3.0:1789760754.563013:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732993524335 time 733028302463 state 0x0 ref 1 00000100:00100000:3.0:1789760754.563025:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515725952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.563074:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23240704, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.563083:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.563100:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.563104:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.563111:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.563124:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.589746:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23240704 -> 23244800 00010000:00010000:0.0:1789760754.589766:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.589782:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.589792:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.589800:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760754.589813:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.589822:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.589840:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.589848:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ed90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.589948:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515725952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26918us (27145us total) trans 4294972991 rc 0/0 00000100:00100000:0.0:1789760754.589959:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5703 00000100:00100000:0.0:1789760754.589969:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732993524335 time 733028302463 state 0x2 ref 0 00000100:00100000:3.0:1789760754.593222:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760754.593506:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726080 00000100:00100000:0.0:1789760754.593516:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726080 00000100:00100000:0.0:1789760754.593582:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 732993524335 time 733028302463 state 0x0 ref 1 00000100:00100000:0.0:1789760754.593598:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733028562879 time 733028302463 state 0x1 ref 1 00000100:00100000:0.0:1789760754.593620:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733028562879 time 733058909441 state 0x0 ref 1 00000100:00100000:0.0:1789760754.593632:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515726080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760754.593680:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23244800, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:0.0:1789760754.593689:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.593720:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.593724:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.593731:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.593745:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.595568:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972991 is committed 00000001:00080000:3.0:1789760754.622079:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972992 is committed 00002000:00010000:2.0:1789760754.622169:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23244800 -> 23248896 00010000:00010000:2.0:1789760754.622185:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.622199:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.622207:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.622215:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.622219:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.622229:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.622247:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.622254:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ed97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.622335:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28701us (29118us total) trans 4294972992 rc 0/0 00000100:00100000:2.0:1789760754.622348:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5704 00000100:00100000:2.0:1789760754.622356:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733028562879 time 733058909441 state 0x2 ref 0 00000100:00100000:1.0:1789760754.626438:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760754.626514:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726208 00000100:00100000:0.0:1789760754.626524:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726208 00000100:00100000:0.0:1789760754.626692:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733028562879 time 733058909441 state 0x0 ref 1 00000100:00100000:0.0:1789760754.626780:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733059169857 time 733058909441 state 0x1 ref 1 00000100:00100000:0.0:1789760754.626916:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733059169857 time 733092202531 state 0x0 ref 1 00000100:00100000:0.0:1789760754.626931:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760754.626985:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23248896, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:0.0:1789760754.626996:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.627012:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.627017:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.627023:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.627036:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.658944:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23248896 -> 23252992 00010000:00010000:3.0:1789760754.658965:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.658978:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.658987:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.658994:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.658999:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.659013:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.659033:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.659041:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ed9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.659135:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515726208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32202us (32702us total) trans 4294972993 rc 0/0 00000100:00100000:3.0:1789760754.659146:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5705 00000100:00100000:3.0:1789760754.659156:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733059169857 time 733092202531 state 0x2 ref 0 00000001:00080000:2.0:1789760754.659288:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972993 is committed 00000100:00100000:2.0:1789760754.662435:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.662515:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726336 00000100:00100000:3.0:1789760754.662534:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726336 00000100:00100000:3.0:1789760754.662584:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733059169857 time 733092202531 state 0x0 ref 1 00000100:00100000:3.0:1789760754.662600:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733092462947 time 733092202531 state 0x1 ref 1 00000100:00100000:3.0:1789760754.662633:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733092462947 time 733127921931 state 0x0 ref 1 00000100:00100000:3.0:1789760754.662645:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515726336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.662691:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23252992, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.662700:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.662746:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.662751:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.662757:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.662771:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.709349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972994 is committed 00002000:00010000:3.0:1789760754.709491:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23252992 -> 23257088 00010000:00010000:3.0:1789760754.709508:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.709522:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.709542:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.709551:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.709556:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.709566:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.709587:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.709596:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1eda5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.709691:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515726336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47042us (47262us total) trans 4294972994 rc 0/0 00000100:00100000:3.0:1789760754.709701:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5706 00000100:00100000:3.0:1789760754.709751:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733092462947 time 733127921931 state 0x2 ref 0 00000100:00100000:1.0:1789760754.713124:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.713207:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726464 00000100:00100000:3.0:1789760754.713217:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726464 00000100:00100000:3.0:1789760754.713271:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733092462947 time 733127921931 state 0x0 ref 1 00000100:00100000:3.0:1789760754.713287:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733128182347 time 733127921931 state 0x1 ref 1 00000100:00100000:3.0:1789760754.713315:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733128182347 time 733178604541 state 0x0 ref 1 00000100:00100000:3.0:1789760754.713328:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515726464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.713376:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23257088, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:3.0:1789760754.713384:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.713400:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.713405:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.713411:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.713431:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.743656:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23257088 -> 23261184 00010000:00010000:3.0:1789760754.743677:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.743691:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.743701:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.743740:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.743746:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.743756:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.743776:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.743785:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.743903:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515726464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30557us (30769us total) trans 4294972995 rc 0/0 00000100:00100000:3.0:1789760754.743913:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5707 00000100:00100000:3.0:1789760754.743923:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733128182347 time 733178604541 state 0x2 ref 0 00000001:00080000:2.0:1789760754.745107:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972995 is committed 00000100:00100000:1.0:1789760754.748146:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.748986:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726592 00000100:00100000:2.0:1789760754.748998:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726592 00000100:00100000:2.0:1789760754.749053:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733128182347 time 733178604541 state 0x0 ref 1 00000100:00100000:2.0:1789760754.749068:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733178864957 time 733178604541 state 0x1 ref 1 00000100:00100000:2.0:1789760754.749098:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733178864957 time 733214387242 state 0x0 ref 1 00000100:00100000:2.0:1789760754.749111:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.749159:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23261184, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760754.749168:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.749183:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.749189:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.749195:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.749209:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.793641:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972996 is committed 00002000:00010000:2.0:1789760754.793768:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23261184 -> 23265280 00010000:00010000:2.0:1789760754.793786:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.793800:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.793912:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.793919:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.793924:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.793934:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.793954:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.793962:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1edb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.794058:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44944us (45917us total) trans 4294972996 rc 0/0 00000100:00100000:2.0:1789760754.794073:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5708 00000100:00100000:2.0:1789760754.794083:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733178864957 time 733214387242 state 0x2 ref 0 00000100:00100000:3.0:1789760754.798023:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760754.798157:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726720 00000100:00100000:1.0:1789760754.798167:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726720 00000100:00100000:1.0:1789760754.798219:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733178864957 time 733214387242 state 0x0 ref 1 00000100:00100000:1.0:1789760754.798233:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733214647658 time 733214387242 state 0x1 ref 1 00000100:00100000:1.0:1789760754.798262:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733214647658 time 733263551901 state 0x0 ref 1 00000100:00100000:1.0:1789760754.798275:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515726720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760754.798324:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23265280, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:1.0:1789760754.798333:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.798355:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.798359:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.798366:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.798379:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.826519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972997 is committed 00002000:00010000:1.0:1789760754.826682:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23265280 -> 23269376 00010000:00010000:1.0:1789760754.826698:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.826738:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.826749:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.826756:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760754.826761:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.826770:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.826790:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.826798:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1edba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.827026:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515726720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28748us (29007us total) trans 4294972997 rc 0/0 00000100:00100000:1.0:1789760754.827037:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5709 00000100:00100000:1.0:1789760754.827046:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733214647658 time 733263551901 state 0x2 ref 0 00000100:00100000:3.0:1789760754.831031:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.831207:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726848 00000100:00100000:2.0:1789760754.831218:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726848 00000100:00100000:2.0:1789760754.831273:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733214647658 time 733263551901 state 0x0 ref 1 00000100:00100000:2.0:1789760754.831289:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733263812317 time 733263551901 state 0x1 ref 1 00000100:00100000:2.0:1789760754.831314:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733263812317 time 733296602988 state 0x0 ref 1 00000100:00100000:2.0:1789760754.831326:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.831373:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23269376, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb807 00010000:00010000:2.0:1789760754.831383:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.831400:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.831405:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.831412:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.831427:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.863414:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972998 is committed 00002000:00010000:2.0:1789760754.863532:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23269376 -> 23273472 00002000:00010000:2.0:1789760754.863537:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:2.0:1789760754.863540:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00010000:00010000:2.0:1789760754.863554:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.863567:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.863576:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.863584:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.863589:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.863599:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.863620:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.863628:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1edc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.863759:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32428us (32735us total) trans 4294972998 rc 0/0 00000100:00100000:2.0:1789760754.863769:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5710 00000100:00100000:2.0:1789760754.863778:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733263812317 time 733296602988 state 0x2 ref 0 00000100:00100000:2.0:1789760754.869254:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.869395:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515726976 00000100:00100000:3.0:1789760754.869405:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515726976 00000100:00100000:3.0:1789760754.869458:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733263812317 time 733296602988 state 0x0 ref 1 00000100:00100000:3.0:1789760754.869474:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733296863404 time 733296602988 state 0x1 ref 1 00000100:00100000:3.0:1789760754.869536:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733296863404 time 733334823826 state 0x0 ref 1 00000100:00100000:3.0:1789760754.869548:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515726976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.869596:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23273472, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760754.869605:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.869620:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.869625:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.869630:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.869650:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760754.900087:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23273472 -> 23277568 00010000:00010000:2.0:1789760754.900106:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.900121:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.900131:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.900138:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760754.900143:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.900153:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.900174:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.900183:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1edc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.900271:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515726976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30719us (31020us total) trans 4294972999 rc 0/0 00000100:00100000:2.0:1789760754.900288:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5711 00000100:00100000:2.0:1789760754.900298:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733296863404 time 733334823826 state 0x2 ref 0 00000001:00080000:0.0:1789760754.900982:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972999 is committed 00000100:00100000:0.0:1789760754.906398:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760754.906748:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727104 00000100:00100000:3.0:1789760754.906757:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727104 00000100:00100000:3.0:1789760754.906833:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733296863404 time 733334823826 state 0x0 ref 1 00000100:00100000:3.0:1789760754.906852:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733335084242 time 733334823826 state 0x1 ref 1 00000100:00100000:3.0:1789760754.906897:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733335084242 time 733372185165 state 0x0 ref 1 00000100:00100000:3.0:1789760754.906912:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515727104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760754.906966:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23277568, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760754.906978:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.906995:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.907000:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.907007:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.907024:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.932971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973000 is committed 00002000:00010000:3.0:1789760754.933183:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23277568 -> 23281664 00010000:00010000:3.0:1789760754.933200:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933214:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933223:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933231:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.933235:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933245:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933266:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.933274:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.933449:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515727104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26535us (27056us total) trans 4294973000 rc 0/0 00000100:00100000:3.0:1789760754.933459:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5712 00000100:00100000:3.0:1789760754.933469:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733335084242 time 733372185165 state 0x2 ref 0 00000100:00100000:3.0:1789760754.937630:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760754.937793:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727232 00000100:00100000:1.0:1789760754.937802:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727232 00000100:00100000:1.0:1789760754.937856:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733335084242 time 733372185165 state 0x0 ref 1 00000100:00100000:1.0:1789760754.937871:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733372445581 time 733372185165 state 0x1 ref 1 00000100:00100000:1.0:1789760754.937908:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733372445581 time 733403197177 state 0x0 ref 1 00000100:00100000:1.0:1789760754.937921:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760754.937967:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23281664, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:1.0:1789760754.937975:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.937992:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.937996:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.938001:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.938012:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.968237:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973001 is committed 00002000:00010000:3.0:1789760754.968342:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23281664 -> 23285760 00010000:00010000:3.0:1789760754.968358:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.968372:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.968382:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.968389:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760754.968394:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.968402:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.968423:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.968431:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1edd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.968528:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30604us (30911us total) trans 4294973001 rc 0/0 00000100:00100000:3.0:1789760754.968543:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5713 00000100:00100000:3.0:1789760754.968553:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733372445581 time 733403197177 state 0x2 ref 0 00000100:00100000:1.0:1789760754.974230:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760754.974378:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727360 00000100:00100000:2.0:1789760754.974389:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727360 00000100:00100000:2.0:1789760754.974444:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733372445581 time 733403197177 state 0x0 ref 1 00000100:00100000:2.0:1789760754.974461:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733403457593 time 733403197177 state 0x1 ref 1 00000100:00100000:2.0:1789760754.974494:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733403457593 time 733439782695 state 0x0 ref 1 00000100:00100000:2.0:1789760754.974506:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515727360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760754.974552:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23285760, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:2.0:1789760754.974561:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.974578:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.974583:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.974589:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.974602:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.006353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973002 is committed 00002000:00010000:2.0:1789760755.006425:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23285760 -> 23289856 00010000:00010000:2.0:1789760755.006441:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.006454:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.006463:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.006470:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760755.006475:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.006484:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.006503:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.006512:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1eddd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.006596:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515727360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32088us (32370us total) trans 4294973002 rc 0/0 00000100:00100000:2.0:1789760755.006607:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5714 00000100:00100000:2.0:1789760755.006616:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733403457593 time 733439782695 state 0x2 ref 0 00010000:00010000:2.0:1789760755.007237:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760755.009634:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760755.009911:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727488 00000100:00100000:0.0:1789760755.009922:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727488 00000100:00100000:1.0:1789760755.010048:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733403457593 time 733439782695 state 0x0 ref 1 00000100:00100000:1.0:1789760755.010064:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733440043111 time 733439782695 state 0x1 ref 1 00000100:00100000:1.0:1789760755.010128:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733440043111 time 733475415760 state 0x0 ref 1 00000100:00100000:1.0:1789760755.010141:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760755.010191:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23289856, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:1.0:1789760755.010200:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.010217:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760755.010222:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.010228:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.010242:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.038902:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973003 is committed 00002000:00010000:3.0:1789760755.039068:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23289856 -> 23293952 00010000:00010000:3.0:1789760755.039084:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.039097:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.039107:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.039114:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.039119:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.039128:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.039149:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.039157:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ede4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.039287:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29144us (29659us total) trans 4294973003 rc 0/0 00000100:00100000:3.0:1789760755.039298:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5715 00000100:00100000:3.0:1789760755.039308:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733440043111 time 733475415760 state 0x2 ref 0 00000100:00100000:2.0:1789760755.046039:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760755.046118:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727616 00000100:00100000:0.0:1789760755.046129:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727616 00000100:00100000:0.0:1789760755.046186:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733440043111 time 733475415760 state 0x0 ref 1 00000100:00100000:0.0:1789760755.046202:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733475676176 time 733475415760 state 0x1 ref 1 00000100:00100000:0.0:1789760755.046234:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733475676176 time 733511522455 state 0x0 ref 1 00000100:00100000:0.0:1789760755.046247:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760755.046297:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23293952, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:0.0:1789760755.046307:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046323:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760755.046328:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.046334:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.046347:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.069518:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23293952 -> 23298048 00010000:00010000:1.0:1789760755.069538:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.069552:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.069562:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.069570:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760755.069575:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.069585:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.069605:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.069614:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1edeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.069703:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515727616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23453us (23670us total) trans 4294973004 rc 0/0 00000100:00100000:1.0:1789760755.069743:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5716 00000100:00100000:1.0:1789760755.069753:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733475676176 time 733511522455 state 0x2 ref 0 00000001:00080000:3.0:1789760755.069834:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973004 is committed 00000100:00100000:1.0:1789760755.073286:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.073369:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727744 00000100:00100000:3.0:1789760755.073379:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727744 00000100:00100000:3.0:1789760755.073435:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733475676176 time 733511522455 state 0x0 ref 1 00000100:00100000:3.0:1789760755.073452:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733511782871 time 733511522455 state 0x1 ref 1 00000100:00100000:3.0:1789760755.073486:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733511782871 time 733538774864 state 0x0 ref 1 00000100:00100000:3.0:1789760755.073498:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.073558:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23298048, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.073567:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.073584:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760755.073588:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.073594:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.073608:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.107150:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973005 is committed 00002000:00010000:3.0:1789760755.107269:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23298048 -> 23302144 00010000:00010000:3.0:1789760755.107284:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.107297:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.107306:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.107314:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.107318:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.107327:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.107348:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.107356:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1edf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.107456:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33955us (34174us total) trans 4294973005 rc 0/0 00000100:00100000:3.0:1789760755.107471:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5717 00000100:00100000:3.0:1789760755.107480:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733511782871 time 733538774864 state 0x2 ref 0 00000100:00100000:0.0:1789760755.111652:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.111779:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515727872 00000100:00100000:3.0:1789760755.111790:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515727872 00000100:00100000:3.0:1789760755.111852:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733511782871 time 733538774864 state 0x0 ref 1 00000100:00100000:3.0:1789760755.111867:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733539035280 time 733538774864 state 0x1 ref 1 00000100:00100000:3.0:1789760755.111902:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733539035280 time 733577190800 state 0x0 ref 1 00000100:00100000:3.0:1789760755.111914:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.111965:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23302144, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.111974:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.111990:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760755.111994:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.112000:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.112013:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.146943:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973006 is committed 00002000:00010000:2.0:1789760755.147339:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23302144 -> 23306240 00010000:00010000:2.0:1789760755.147356:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.147370:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.147381:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.147389:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760755.147394:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.147403:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.147426:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.147434:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1edf9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.147539:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515727872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35622us (35892us total) trans 4294973006 rc 0/0 00000100:00100000:2.0:1789760755.147553:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5718 00000100:00100000:2.0:1789760755.147562:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733539035280 time 733577190800 state 0x2 ref 0 00000100:00100000:0.0:1789760755.151247:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.151331:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728000 00000100:00100000:3.0:1789760755.151342:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728000 00000100:00100000:3.0:1789760755.151394:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733539035280 time 733577190800 state 0x0 ref 1 00000100:00100000:3.0:1789760755.151410:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733577451216 time 733577190800 state 0x1 ref 1 00000100:00100000:3.0:1789760755.151440:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733577451216 time 733616729571 state 0x0 ref 1 00000100:00100000:3.0:1789760755.151453:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515728000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.151501:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23306240, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.151509:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.151536:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760755.151541:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.151548:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.151562:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.179727:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23306240 -> 23310336 00010000:00010000:3.0:1789760755.179749:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.179762:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.179772:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.179780:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.179785:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.179921:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.179942:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.179951:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.180128:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515728000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28671us (28885us total) trans 4294973007 rc 0/0 00000100:00100000:3.0:1789760755.180138:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5719 00000100:00100000:3.0:1789760755.180149:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733577451216 time 733616729571 state 0x2 ref 0 00000001:00080000:3.0:1789760755.180188:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973007 is committed 00000100:00100000:1.0:1789760755.184506:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760755.184656:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728128 00000100:00100000:0.0:1789760755.184666:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728128 00000100:00100000:0.0:1789760755.184975:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733577451216 time 733616729571 state 0x0 ref 1 00000100:00100000:0.0:1789760755.184992:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733616989987 time 733616729571 state 0x1 ref 1 00000100:00100000:0.0:1789760755.185053:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733616989987 time 733650340760 state 0x0 ref 1 00000100:00100000:0.0:1789760755.185067:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760755.185115:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23310336, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:0.0:1789760755.185124:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.185140:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760755.185145:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.185151:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.185165:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.229696:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973008 is committed 00002000:00010000:0.0:1789760755.230036:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23310336 -> 23314432 00010000:00010000:0.0:1789760755.230052:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.230066:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.230075:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.230084:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760755.230088:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.230098:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.230118:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.230127:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.230259:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45190us (45756us total) trans 4294973008 rc 0/0 00000100:00100000:0.0:1789760755.230271:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5720 00000100:00100000:0.0:1789760755.230280:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733616989987 time 733650340760 state 0x2 ref 0 00000100:00100000:0.0:1789760755.233591:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.233684:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728256 00000100:00100000:3.0:1789760755.233695:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728256 00000100:00100000:2.0:1789760755.234077:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733616989987 time 733650340760 state 0x0 ref 1 00000100:00100000:2.0:1789760755.234095:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733650601176 time 733650340760 state 0x1 ref 1 00000100:00100000:2.0:1789760755.234126:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733650601176 time 733699414408 state 0x0 ref 1 00000100:00100000:2.0:1789760755.234140:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760755.234192:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23314432, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:2.0:1789760755.234201:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.234216:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760755.234220:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.234226:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.234240:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.271134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973009 is committed 00002000:00010000:3.0:1789760755.272176:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23314432 -> 23318528 00010000:00010000:3.0:1789760755.272195:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272212:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272220:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272237:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.272242:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272251:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272270:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.272278:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1ee0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.272374:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38231us (38844us total) trans 4294973009 rc 0/0 00000100:00100000:3.0:1789760755.272386:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5721 00000100:00100000:3.0:1789760755.272396:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733650601176 time 733699414408 state 0x2 ref 0 00000100:00100000:2.0:1789760755.276295:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760755.276457:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728384 00000100:00100000:1.0:1789760755.276468:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728384 00000100:00100000:1.0:1789760755.276696:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733650601176 time 733699414408 state 0x0 ref 1 00000100:00100000:1.0:1789760755.276756:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733699674824 time 733699414408 state 0x1 ref 1 00000100:00100000:1.0:1789760755.276783:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733699674824 time 733742071861 state 0x0 ref 1 00000100:00100000:1.0:1789760755.276980:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515728384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760755.277030:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23318528, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:1.0:1789760755.277040:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.277056:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760755.277061:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.277068:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.277083:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.324639:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23318528 -> 23322624 00010000:00010000:3.0:1789760755.324661:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.324675:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.324684:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.324692:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.324697:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.324739:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.324759:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.324767:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1ee15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.325029:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515728384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48230us (48738us total) trans 4294973010 rc 0/0 00000100:00100000:2.0:1789760755.325043:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5722 00000100:00100000:2.0:1789760755.325054:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733699674824 time 733742071861 state 0x2 ref 0 00000001:00080000:0.0:1789760755.326136:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973010 is committed 00000100:00100000:2.0:1789760755.328601:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.328851:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728512 00000100:00100000:3.0:1789760755.328862:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728512 00000100:00100000:3.0:1789760755.328925:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733699674824 time 733742071861 state 0x0 ref 1 00000100:00100000:3.0:1789760755.328940:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733742332277 time 733742071861 state 0x1 ref 1 00000100:00100000:3.0:1789760755.328966:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733742332277 time 733794255303 state 0x0 ref 1 00000100:00100000:3.0:1789760755.328981:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.329032:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23322624, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.329042:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.329059:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760755.329063:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.329069:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.329084:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.363123:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973011 is committed 00002000:00010000:3.0:1789760755.363237:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23322624 -> 23326720 00010000:00010000:3.0:1789760755.363253:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.363267:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.363277:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.363284:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.363291:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.363300:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.363319:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.363328:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.363421:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34439us (34825us total) trans 4294973011 rc 0/0 00000100:00100000:3.0:1789760755.363431:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5723 00000100:00100000:3.0:1789760755.363441:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733742332277 time 733794255303 state 0x2 ref 0 00000100:00100000:1.0:1789760755.367963:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.368037:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728640 00000100:00100000:3.0:1789760755.368047:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728640 00000100:00100000:3.0:1789760755.368102:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733742332277 time 733794255303 state 0x0 ref 1 00000100:00100000:3.0:1789760755.368117:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733794515719 time 733794255303 state 0x1 ref 1 00000100:00100000:3.0:1789760755.368149:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733794515719 time 733833438186 state 0x0 ref 1 00000100:00100000:3.0:1789760755.368162:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.368213:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23326720, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.368223:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.368239:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760755.368244:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.368250:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.368263:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.394773:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23326720 -> 23330816 00010000:00010000:3.0:1789760755.394793:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.394815:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.394824:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.394831:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.394836:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.394846:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.394873:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.394891:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.394990:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515728640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26824us (27031us total) trans 4294973012 rc 0/0 00000100:00100000:3.0:1789760755.395000:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5724 00000100:00100000:3.0:1789760755.395009:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733794515719 time 733833438186 state 0x2 ref 0 00000001:00080000:1.0:1789760755.396119:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973012 is committed 00000100:00100000:2.0:1789760755.400299:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.400392:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728768 00000100:00100000:3.0:1789760755.400402:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728768 00000100:00100000:3.0:1789760755.400455:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733794515719 time 733833438186 state 0x0 ref 1 00000100:00100000:3.0:1789760755.400468:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733833698602 time 733833438186 state 0x1 ref 1 00000100:00100000:3.0:1789760755.400542:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733833698602 time 733865830357 state 0x0 ref 1 00000100:00100000:3.0:1789760755.400555:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515728768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.400608:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23330816, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.400618:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.400635:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760755.400700:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.400737:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.400749:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.428864:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23330816 -> 23334912 00010000:00010000:1.0:1789760755.428896:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.428911:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.428921:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.428929:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760755.428934:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.428944:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.428966:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.428974:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ee2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.429072:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515728768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28515us (28777us total) trans 4294973013 rc 0/0 00000100:00100000:1.0:1789760755.429083:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5725 00000100:00100000:1.0:1789760755.429093:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733833698602 time 733865830357 state 0x2 ref 0 00000001:00080000:2.0:1789760755.429364:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973013 is committed 00000100:00100000:2.0:1789760755.433093:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760755.433311:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515728896 00000100:00100000:0.0:1789760755.433322:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515728896 00000100:00100000:0.0:1789760755.433377:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733833698602 time 733865830357 state 0x0 ref 1 00000100:00100000:0.0:1789760755.433393:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733866090773 time 733865830357 state 0x1 ref 1 00000100:00100000:0.0:1789760755.433420:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733866090773 time 733898708727 state 0x0 ref 1 00000100:00100000:0.0:1789760755.433432:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760755.433478:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23334912, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:0.0:1789760755.433487:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.433502:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760755.433507:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.433513:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.433528:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.486019:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973014 is committed 00002000:00010000:3.0:1789760755.486165:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23334912 -> 23339008 00010000:00010000:3.0:1789760755.486181:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.486194:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.486203:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.486211:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.486217:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.486227:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.486246:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.486254:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ee31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.486375:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515728896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52940us (53287us total) trans 4294973014 rc 0/0 00000100:00100000:3.0:1789760755.486385:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5726 00000100:00100000:3.0:1789760755.486395:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733866090773 time 733898708727 state 0x2 ref 0 00000100:00100000:3.0:1789760755.489497:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760755.489561:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729024 00000100:00100000:2.0:1789760755.489571:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729024 00000100:00100000:2.0:1789760755.489633:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733866090773 time 733898708727 state 0x0 ref 1 00000100:00100000:2.0:1789760755.489648:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733898969143 time 733898708727 state 0x1 ref 1 00000100:00100000:2.0:1789760755.489676:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733898969143 time 733954964644 state 0x0 ref 1 00000100:00100000:2.0:1789760755.489690:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515729024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760755.490233:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23339008, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:2.0:1789760755.490242:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.490257:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760755.490261:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.490267:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.490285:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.522775:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973015 is committed 00002000:00010000:3.0:1789760755.522977:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23339008 -> 23343104 00010000:00010000:3.0:1789760755.522995:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.523008:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.523018:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.523025:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.523029:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.523039:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.523056:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.523069:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1ee38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.523201:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515729024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33510us (33708us total) trans 4294973015 rc 0/0 00000100:00100000:3.0:1789760755.523211:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5727 00000100:00100000:3.0:1789760755.523221:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733898969143 time 733954964644 state 0x2 ref 0 00000100:00100000:2.0:1789760755.529196:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760755.529295:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729152 00000100:00100000:1.0:1789760755.529305:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729152 00000100:00100000:1.0:1789760755.529364:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733898969143 time 733954964644 state 0x0 ref 1 00000100:00100000:1.0:1789760755.529381:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733955225060 time 733954964644 state 0x1 ref 1 00000100:00100000:1.0:1789760755.529407:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733955225060 time 733994697127 state 0x0 ref 1 00000100:00100000:1.0:1789760755.529419:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760755.529474:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23343104, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:1.0:1789760755.529484:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.529501:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760755.529505:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.529511:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.529524:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.559937:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23343104 -> 23347200 00010000:00010000:3.0:1789760755.559960:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.559974:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.559983:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.559990:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.559995:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.560004:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.560032:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.560041:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ee3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.560138:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515729152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30715us (30946us total) trans 4294973016 rc 0/0 00000100:00100000:3.0:1789760755.560149:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5728 00000100:00100000:3.0:1789760755.560159:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733955225060 time 733994697127 state 0x2 ref 0 00000001:00080000:0.0:1789760755.560325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973016 is committed 00000100:00100000:2.0:1789760755.567439:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.567527:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729280 00000100:00100000:3.0:1789760755.567538:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729280 00000100:00100000:3.0:1789760755.567593:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733955225060 time 733994697127 state 0x0 ref 1 00000100:00100000:3.0:1789760755.567608:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733994957543 time 733994697127 state 0x1 ref 1 00000100:00100000:3.0:1789760755.567640:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733994957543 time 734032928907 state 0x0 ref 1 00000100:00100000:3.0:1789760755.567651:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.567694:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23347200, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.567745:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.567767:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.567773:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.567779:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.567793:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.593514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973017 is committed 00002000:00010000:3.0:1789760755.593750:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23347200 -> 23351296 00010000:00010000:3.0:1789760755.593766:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.593779:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.593790:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.593895:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.593901:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.593911:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.593930:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.593943:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.594036:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26381us (26602us total) trans 4294973017 rc 0/0 00000100:00100000:3.0:1789760755.594047:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5729 00000100:00100000:3.0:1789760755.594057:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733994957543 time 734032928907 state 0x2 ref 0 00000100:00100000:2.0:1789760755.600382:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760755.600490:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729408 00000100:00100000:0.0:1789760755.600500:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729408 00000100:00100000:0.0:1789760755.600553:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 733994957543 time 734032928907 state 0x0 ref 1 00000100:00100000:0.0:1789760755.600568:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734033189323 time 734032928907 state 0x1 ref 1 00000100:00100000:0.0:1789760755.600730:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734033189323 time 734065993804 state 0x0 ref 1 00000100:00100000:0.0:1789760755.600746:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760755.600807:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23351296, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:0.0:1789760755.600816:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.600836:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.600841:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.600848:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.600867:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.631431:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23351296 -> 23355392 00010000:00010000:3.0:1789760755.631451:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.631464:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.631474:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.631482:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.631487:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.631497:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.631523:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.631532:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ee4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.631631:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515729408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30882us (31253us total) trans 4294973018 rc 0/0 00000100:00100000:3.0:1789760755.631641:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5730 00000100:00100000:3.0:1789760755.631652:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734033189323 time 734065993804 state 0x2 ref 0 00000001:00080000:2.0:1789760755.632317:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973018 is committed 00000100:00100000:2.0:1789760755.636974:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.637044:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729536 00000100:00100000:3.0:1789760755.637054:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729536 00000100:00100000:3.0:1789760755.637107:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734033189323 time 734065993804 state 0x0 ref 1 00000100:00100000:3.0:1789760755.637123:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734066254220 time 734065993804 state 0x1 ref 1 00000100:00100000:3.0:1789760755.637153:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734066254220 time 734102441601 state 0x0 ref 1 00000100:00100000:3.0:1789760755.637167:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.637218:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23355392, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.637228:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.637243:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.637248:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.637255:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.637269:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.659118:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973019 is committed 00002000:00010000:3.0:1789760755.659307:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23355392 -> 23359488 00010000:00010000:3.0:1789760755.659323:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.659335:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.659345:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.659354:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.659360:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.659370:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.659390:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.659399:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.659496:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22327us (22527us total) trans 4294973019 rc 0/0 00000100:00100000:3.0:1789760755.659507:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5731 00000100:00100000:3.0:1789760755.659516:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734066254220 time 734102441601 state 0x2 ref 0 00000100:00100000:0.0:1789760755.668548:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.669076:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729664 00000100:00100000:3.0:1789760755.669085:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729664 00000100:00100000:3.0:1789760755.669139:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734066254220 time 734102441601 state 0x0 ref 1 00000100:00100000:3.0:1789760755.669164:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734102702017 time 734102441601 state 0x1 ref 1 00000100:00100000:3.0:1789760755.669293:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734102702017 time 734134580752 state 0x0 ref 1 00000100:00100000:3.0:1789760755.669305:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.669351:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23359488, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.669359:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.669375:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.669380:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.669386:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.669400:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.722621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973020 is committed 00002000:00010000:1.0:1789760755.722704:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23359488 -> 23363584 00010000:00010000:1.0:1789760755.722750:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.722765:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.722774:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.722781:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760755.722800:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.722810:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.722830:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.722837:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1ee5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.722986:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515729664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53676us (54442us total) trans 4294973020 rc 0/0 00000100:00100000:0.0:1789760755.722998:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5732 00000100:00100000:0.0:1789760755.723008:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734102702017 time 734134580752 state 0x2 ref 0 00000100:00100000:0.0:1789760755.728201:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760755.728264:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729792 00000100:00100000:2.0:1789760755.728273:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729792 00000100:00100000:2.0:1789760755.728322:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734102702017 time 734134580752 state 0x0 ref 1 00000100:00100000:2.0:1789760755.728336:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734134841168 time 734134580752 state 0x1 ref 1 00000100:00100000:2.0:1789760755.728414:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734134841168 time 734193701950 state 0x0 ref 1 00000100:00100000:2.0:1789760755.728427:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515729792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760755.728476:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23363584, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:2.0:1789760755.728485:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.728500:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760755.728505:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.728510:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.728525:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.770629:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973021 is committed 00002000:00010000:0.0:1789760755.772587:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23363584 -> 23367680 00010000:00010000:0.0:1789760755.772607:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.772622:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.772631:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.772639:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760755.772644:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.772661:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.772680:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.772688:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1ee62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.772864:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515729792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44433us (44666us total) trans 4294973021 rc 0/0 00000100:00100000:0.0:1789760755.772883:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5733 00000100:00100000:0.0:1789760755.772893:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734134841168 time 734193701950 state 0x2 ref 0 00000100:00100000:2.0:1789760755.776350:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760755.776511:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515729920 00000100:00100000:1.0:1789760755.776532:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515729920 00000100:00100000:1.0:1789760755.776587:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734134841168 time 734193701950 state 0x0 ref 1 00000100:00100000:1.0:1789760755.776601:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734193962366 time 734193701950 state 0x1 ref 1 00000100:00100000:1.0:1789760755.776631:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734193962366 time 734241920380 state 0x0 ref 1 00000100:00100000:1.0:1789760755.776644:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515729920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760755.776697:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23367680, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:1.0:1789760755.776738:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.776755:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760755.776759:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.776766:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.776779:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.818115:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973022 is committed 00002000:00010000:1.0:1789760755.818175:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23367680 -> 23371776 00010000:00010000:1.0:1789760755.818190:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.818202:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.818217:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.818224:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760755.818229:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.818238:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.818357:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.818366:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.818452:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515729920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41805us (42108us total) trans 4294973022 rc 0/0 00000100:00100000:1.0:1789760755.818461:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5734 00000100:00100000:1.0:1789760755.818470:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734193962366 time 734241920380 state 0x2 ref 0 00000100:00100000:2.0:1789760755.821966:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.822094:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730048 00000100:00100000:3.0:1789760755.822106:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730048 00000100:00100000:3.0:1789760755.822163:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734193962366 time 734241920380 state 0x0 ref 1 00000100:00100000:3.0:1789760755.822179:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734242180796 time 734241920380 state 0x1 ref 1 00000100:00100000:3.0:1789760755.822211:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734242180796 time 734287499945 state 0x0 ref 1 00000100:00100000:3.0:1789760755.822223:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515730048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.822282:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23371776, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb807 00010000:00010000:3.0:1789760755.822292:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.822307:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.822312:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.822319:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.822333:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.857390:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23371776 -> 23375872 00002000:00010000:1.0:1789760755.857397:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:1.0:1789760755.857400:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00010000:00010000:1.0:1789760755.857412:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.857426:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.857435:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.857442:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760755.857447:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.857456:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.857474:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.857482:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ee70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.857571:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515730048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35344us (35609us total) trans 4294973023 rc 0/0 00000100:00100000:1.0:1789760755.857581:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5735 00000100:00100000:1.0:1789760755.857591:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734242180796 time 734287499945 state 0x2 ref 0 00000001:00080000:1.0:1789760755.857627:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973023 is committed 00000100:00100000:2.0:1789760755.860510:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.860559:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730176 00000100:00100000:3.0:1789760755.860568:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730176 00000100:00100000:3.0:1789760755.860624:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734242180796 time 734287499945 state 0x0 ref 1 00000100:00100000:3.0:1789760755.860639:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734287760361 time 734287499945 state 0x1 ref 1 00000100:00100000:3.0:1789760755.860671:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734287760361 time 734325961026 state 0x0 ref 1 00000100:00100000:3.0:1789760755.860683:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515730176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.860748:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23375872, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760755.860757:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.860772:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.860777:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.860783:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.860807:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.894854:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23375872 -> 23379968 00010000:00010000:3.0:1789760755.894884:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.894897:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.894906:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.894914:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.894920:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.894929:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.894947:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.894955:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.895043:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515730176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34358us (34537us total) trans 4294973024 rc 0/0 00000100:00100000:3.0:1789760755.895054:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5736 00000100:00100000:3.0:1789760755.895063:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734287760361 time 734325961026 state 0x2 ref 0 00000001:00080000:1.0:1789760755.895184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973024 is committed 00000100:00100000:2.0:1789760755.898921:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.899022:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730304 00000100:00100000:3.0:1789760755.899034:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730304 00000100:00100000:3.0:1789760755.899086:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734287760361 time 734325961026 state 0x0 ref 1 00000100:00100000:3.0:1789760755.899102:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734326221442 time 734325961026 state 0x1 ref 1 00000100:00100000:3.0:1789760755.899135:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734326221442 time 734364424882 state 0x0 ref 1 00000100:00100000:3.0:1789760755.899148:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515730304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.899358:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23379968, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760755.899368:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.899383:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.899388:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.899395:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.899408:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.924335:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973025 is committed 00002000:00010000:3.0:1789760755.924435:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23379968 -> 23384064 00010000:00010000:3.0:1789760755.924451:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.924464:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.924473:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.924480:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.924485:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.924495:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.924519:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.924528:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ee7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.924616:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515730304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25465us (25698us total) trans 4294973025 rc 0/0 00000100:00100000:3.0:1789760755.924626:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5737 00000100:00100000:3.0:1789760755.924636:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734326221442 time 734364424882 state 0x2 ref 0 00000100:00100000:0.0:1789760755.929096:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760755.929193:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730432 00000100:00100000:3.0:1789760755.929203:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730432 00000100:00100000:3.0:1789760755.929257:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734326221442 time 734364424882 state 0x0 ref 1 00000100:00100000:3.0:1789760755.929336:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734364685298 time 734364424882 state 0x1 ref 1 00000100:00100000:3.0:1789760755.929369:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734364685298 time 734394657933 state 0x0 ref 1 00000100:00100000:3.0:1789760755.929381:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515730432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760755.929433:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23384064, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760755.929442:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.929457:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.929462:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.929469:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.929483:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.978055:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973026 is committed 00002000:00010000:3.0:1789760755.978267:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23384064 -> 23388160 00010000:00010000:3.0:1789760755.978283:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.978296:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.978306:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.978313:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760755.978318:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.978326:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.978344:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.978359:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ee85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.978455:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515730432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49071us (49364us total) trans 4294973026 rc 0/0 00000100:00100000:3.0:1789760755.978466:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5738 00000100:00100000:3.0:1789760755.978476:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734364685298 time 734394657933 state 0x2 ref 0 00000100:00100000:2.0:1789760755.983109:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760755.983198:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730560 00000100:00100000:0.0:1789760755.983208:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730560 00000100:00100000:0.0:1789760755.983263:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734364685298 time 734394657933 state 0x0 ref 1 00000100:00100000:0.0:1789760755.983279:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734394918349 time 734394657933 state 0x1 ref 1 00000100:00100000:0.0:1789760755.983307:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734394918349 time 734448596392 state 0x0 ref 1 00000100:00100000:0.0:1789760755.983319:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515730560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760755.983371:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23388160, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:0.0:1789760755.983381:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.983396:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.983400:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.983406:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.983420:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.034245:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760756.049453:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973027 is committed 00002000:00010000:2.0:1789760756.049637:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23388160 -> 23392256 00010000:00010000:2.0:1789760756.049653:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.049665:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.049674:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.049681:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760756.049686:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.049694:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.049747:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.049756:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1ee8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.049855:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515730560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66533us (66751us total) trans 4294973027 rc 0/0 00000100:00100000:2.0:1789760756.049865:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5739 00000100:00100000:2.0:1789760756.049883:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734394918349 time 734448596392 state 0x2 ref 0 00000100:00100000:0.0:1789760756.053691:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760756.053896:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730688 00000100:00100000:1.0:1789760756.053905:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730688 00000100:00100000:1.0:1789760756.053996:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734394918349 time 734448596392 state 0x0 ref 1 00000100:00100000:1.0:1789760756.054012:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734448856808 time 734448596392 state 0x1 ref 1 00000100:00100000:1.0:1789760756.054076:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734448856808 time 734519365220 state 0x0 ref 1 00000100:00100000:1.0:1789760756.054090:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515730688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760756.054134:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23392256, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:1.0:1789760756.054142:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.054157:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760756.054162:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.054169:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.054182:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760756.054680:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760756.083546:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973028 is committed 00002000:00010000:1.0:1789760756.083760:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23392256 -> 23396352 00010000:00010000:1.0:1789760756.083775:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.083848:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.083858:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.083866:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760756.083870:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.083890:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.083910:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.083918:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ee93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.084010:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515730688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29917us (30322us total) trans 4294973028 rc 0/0 00000100:00100000:1.0:1789760756.084066:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5740 00000100:00100000:1.0:1789760756.084076:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734448856808 time 734519365220 state 0x2 ref 0 00000100:00100000:0.0:1789760756.087672:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760756.087894:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730816 00000100:00100000:1.0:1789760756.087903:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730816 00000100:00100000:1.0:1789760756.087956:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734448856808 time 734519365220 state 0x0 ref 1 00000100:00100000:1.0:1789760756.087970:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734519625636 time 734519365220 state 0x1 ref 1 00000100:00100000:1.0:1789760756.088037:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734519625636 time 734553325793 state 0x0 ref 1 00000100:00100000:1.0:1789760756.088050:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515730816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760756.088095:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23396352, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:1.0:1789760756.088104:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.088119:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760756.088123:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.088129:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.088144:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.118607:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23396352 -> 23400448 00010000:00010000:2.0:1789760756.118628:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.118641:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.118651:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.118659:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760756.118664:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.118674:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.118699:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.118728:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ee9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.118906:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515730816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30852us (31237us total) trans 4294973029 rc 0/0 00000100:00100000:2.0:1789760756.118918:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5741 00000100:00100000:2.0:1789760756.118927:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734519625636 time 734553325793 state 0x2 ref 0 00000001:00080000:0.0:1789760756.119841:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973029 is committed 00000100:00100000:0.0:1789760756.125792:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760756.125937:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515730944 00000100:00100000:3.0:1789760756.125947:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515730944 00000100:00100000:3.0:1789760756.126002:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734519625636 time 734553325793 state 0x0 ref 1 00000100:00100000:3.0:1789760756.126017:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734553586209 time 734553325793 state 0x1 ref 1 00000100:00100000:3.0:1789760756.126042:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734553586209 time 734591331282 state 0x0 ref 1 00000100:00100000:3.0:1789760756.126055:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515730944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760756.126104:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23400448, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760756.126113:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.126129:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760756.126134:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.126140:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.126160:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.201935:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973030 is committed 00002000:00010000:3.0:1789760756.202049:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23400448 -> 23404544 00010000:00010000:3.0:1789760756.202064:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.202078:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.202087:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.202095:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760756.202101:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.202110:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.202131:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.202140:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.202235:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515730944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 76178us (76448us total) trans 4294973030 rc 0/0 00000100:00100000:3.0:1789760756.202246:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5742 00000100:00100000:3.0:1789760756.202256:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734553586209 time 734591331282 state 0x2 ref 0 00000100:00100000:1.0:1789760756.210518:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760756.210897:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731072 00000100:00100000:3.0:1789760756.210906:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731072 00000100:00100000:3.0:1789760756.210964:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734553586209 time 734591331282 state 0x0 ref 1 00000100:00100000:3.0:1789760756.210980:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734591591698 time 734591331282 state 0x1 ref 1 00000100:00100000:3.0:1789760756.211012:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734591591698 time 734676301305 state 0x0 ref 1 00000100:00100000:3.0:1789760756.211025:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515731072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760756.211079:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23404544, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760756.211087:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.211102:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760756.211107:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.211114:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.211128:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.279973:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973031 is committed 00002000:00010000:3.0:1789760756.281301:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23404544 -> 23408640 00010000:00010000:3.0:1789760756.281320:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.281408:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.281420:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.281428:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760756.281433:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.281443:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.281464:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.281471:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eea8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.281626:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515731072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70596us (71111us total) trans 4294973031 rc 0/0 00000100:00100000:3.0:1789760756.281641:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5743 00000100:00100000:3.0:1789760756.281651:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734591591698 time 734676301305 state 0x2 ref 0 00000100:00100000:1.0:1789760756.291691:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760756.291908:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731200 00000100:00100000:3.0:1789760756.291918:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731200 00000100:00100000:3.0:1789760756.291968:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734591591698 time 734676301305 state 0x0 ref 1 00000100:00100000:3.0:1789760756.291983:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734676561721 time 734676301305 state 0x1 ref 1 00000100:00100000:3.0:1789760756.292013:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734676561721 time 734757301788 state 0x0 ref 1 00000100:00100000:3.0:1789760756.292025:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515731200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760756.292074:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23408640, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760756.292082:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.292098:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760756.292103:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.292109:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.292124:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.360844:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973032 is committed 00002000:00010000:0.0:1789760756.360970:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23408640 -> 23412736 00010000:00010000:0.0:1789760756.360984:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.360998:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.361007:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.361016:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760756.361021:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.361030:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.361049:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.361057:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1eeaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.361152:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515731200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69124us (69466us total) trans 4294973032 rc 0/0 00000100:00100000:0.0:1789760756.361161:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5744 00000100:00100000:0.0:1789760756.361171:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734676561721 time 734757301788 state 0x2 ref 0 00000100:00100000:2.0:1789760756.364459:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760756.364532:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731328 00000100:00100000:0.0:1789760756.364542:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731328 00000100:00100000:0.0:1789760756.364594:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734676561721 time 734757301788 state 0x0 ref 1 00000100:00100000:0.0:1789760756.364618:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734757562204 time 734757301788 state 0x1 ref 1 00000100:00100000:0.0:1789760756.364645:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734757562204 time 734829934570 state 0x0 ref 1 00000100:00100000:0.0:1789760756.364660:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515731328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760756.364743:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23412736, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:0.0:1789760756.364752:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.364767:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760756.364771:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.364778:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.364791:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.433777:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973033 is committed 00002000:00010000:3.0:1789760756.434470:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23412736 -> 23416832 00010000:00010000:3.0:1789760756.434488:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.434501:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.434510:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.434519:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760756.434524:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.434534:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.434555:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.434563:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eeb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.434665:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515731328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70001us (70209us total) trans 4294973033 rc 0/0 00000100:00100000:3.0:1789760756.434677:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5745 00000100:00100000:3.0:1789760756.434687:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734757562204 time 734829934570 state 0x2 ref 0 00000100:00100000:3.0:1789760756.450130:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760756.450248:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731456 00000100:00100000:1.0:1789760756.450259:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731456 00000100:00100000:1.0:1789760756.450311:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734757562204 time 734829934570 state 0x0 ref 1 00000100:00100000:1.0:1789760756.450326:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734830194986 time 734829934570 state 0x1 ref 1 00000100:00100000:1.0:1789760756.450419:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734830194986 time 734915706765 state 0x0 ref 1 00000100:00100000:1.0:1789760756.450433:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515731456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760756.450479:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23416832, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:1.0:1789760756.450489:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.450503:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760756.450508:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.450514:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.450528:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.482604:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23416832 -> 23420928 00010000:00010000:2.0:1789760756.482625:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.482640:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.482649:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.482658:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760756.482663:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.482675:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.482694:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.482702:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eebd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.482825:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515731456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32390us (32700us total) trans 4294973034 rc 0/0 00000100:00100000:2.0:1789760756.482834:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5746 00000100:00100000:2.0:1789760756.482843:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734830194986 time 734915706765 state 0x2 ref 0 00000001:00080000:3.0:1789760756.483467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973034 is committed 00000100:00100000:1.0:1789760756.493190:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760756.493309:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731584 00000100:00100000:2.0:1789760756.493320:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731584 00000100:00100000:2.0:1789760756.493473:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734830194986 time 734915706765 state 0x0 ref 1 00000100:00100000:2.0:1789760756.493489:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734915967181 time 734915706765 state 0x1 ref 1 00000100:00100000:2.0:1789760756.493510:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734915967181 time 734958799582 state 0x0 ref 1 00000100:00100000:2.0:1789760756.493522:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515731584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760756.493571:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23420928, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:2.0:1789760756.493580:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.493596:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760756.493601:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.493607:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.493621:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.554477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973035 is committed 00002000:00010000:3.0:1789760756.554628:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23420928 -> 23425024 00010000:00010000:3.0:1789760756.554643:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.554657:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.554667:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.554674:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760756.554679:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.554688:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.554741:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.554750:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1eec4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.554914:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515731584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61389us (61728us total) trans 4294973035 rc 0/0 00000100:00100000:2.0:1789760756.554924:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5747 00000100:00100000:2.0:1789760756.554933:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734915967181 time 734958799582 state 0x2 ref 0 00000100:00100000:1.0:1789760756.563011:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760756.563099:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731712 00000100:00100000:3.0:1789760756.563109:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731712 00000100:00100000:3.0:1789760756.563161:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734915967181 time 734958799582 state 0x0 ref 1 00000100:00100000:3.0:1789760756.563237:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734959059998 time 734958799582 state 0x1 ref 1 00000100:00100000:3.0:1789760756.563270:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734959059998 time 735028558732 state 0x0 ref 1 00000100:00100000:3.0:1789760756.563283:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515731712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760756.563343:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23425024, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:3.0:1789760756.563353:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.563368:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.563373:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.563379:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.563393:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.610977:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23425024 -> 23429120 00010000:00010000:1.0:1789760756.610998:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.611011:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.611024:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.611031:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760756.611035:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.611044:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.611064:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.611073:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1eecb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.611167:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515731712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47881us (48160us total) trans 4294973036 rc 0/0 00000100:00100000:1.0:1789760756.611178:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5748 00000100:00100000:1.0:1789760756.611188:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734959059998 time 735028558732 state 0x2 ref 0 00000001:00080000:2.0:1789760756.611479:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973036 is committed 00000100:00100000:3.0:1789760756.621195:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760756.621304:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731840 00000100:00100000:0.0:1789760756.621315:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731840 00000100:00100000:0.0:1789760756.621369:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 734959059998 time 735028558732 state 0x0 ref 1 00000100:00100000:0.0:1789760756.621384:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735028819148 time 735028558732 state 0x1 ref 1 00000100:00100000:0.0:1789760756.621415:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735028819148 time 735086703541 state 0x0 ref 1 00000100:00100000:0.0:1789760756.621427:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515731840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760756.621475:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23429120, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:0.0:1789760756.621483:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.621505:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.621511:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.621517:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.621532:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760756.659411:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23429120 -> 23433216 00010000:00010000:0.0:1789760756.659430:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.659442:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.659451:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.659457:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760756.659462:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.659470:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.659488:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.659495:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.659576:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515731840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38147us (38391us total) trans 4294973037 rc 0/0 00000100:00100000:0.0:1789760756.659585:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5749 00000100:00100000:0.0:1789760756.659594:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735028819148 time 735086703541 state 0x2 ref 0 00000001:00080000:0.0:1789760756.659631:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973037 is committed 00000100:00100000:3.0:1789760756.666914:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760756.667050:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515731968 00000100:00100000:0.0:1789760756.667060:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515731968 00000100:00100000:0.0:1789760756.667114:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735028819148 time 735086703541 state 0x0 ref 1 00000100:00100000:0.0:1789760756.667128:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735086963957 time 735086703541 state 0x1 ref 1 00000100:00100000:0.0:1789760756.667258:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735086963957 time 735132545968 state 0x0 ref 1 00000100:00100000:0.0:1789760756.667273:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515731968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760756.667320:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23433216, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:0.0:1789760756.667330:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.667347:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.667351:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.667357:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.667371:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.702831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973038 is committed 00002000:00010000:0.0:1789760756.703381:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23433216 -> 23437312 00010000:00010000:0.0:1789760756.703397:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.703412:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.703421:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.703429:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760756.703434:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.703442:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.703462:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.703470:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1eed9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.703570:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515731968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36295us (36662us total) trans 4294973038 rc 0/0 00000100:00100000:0.0:1789760756.703582:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5750 00000100:00100000:0.0:1789760756.703592:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735086963957 time 735132545968 state 0x2 ref 0 00000100:00100000:0.0:1789760756.712152:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760756.712402:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732096 00000100:00100000:1.0:1789760756.712413:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732096 00000100:00100000:1.0:1789760756.712466:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735086963957 time 735132545968 state 0x0 ref 1 00000100:00100000:1.0:1789760756.712491:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735132806384 time 735132545968 state 0x1 ref 1 00000100:00100000:1.0:1789760756.712556:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735132806384 time 735177844677 state 0x0 ref 1 00000100:00100000:1.0:1789760756.712568:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515732096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760756.712618:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23437312, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:1.0:1789760756.712627:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.712642:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.712647:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.712653:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.712666:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760756.743187:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23437312 -> 23441408 00010000:00010000:0.0:1789760756.743208:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.743221:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.743231:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.743238:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760756.743243:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.743253:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.743273:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.743286:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1eee0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.743388:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515732096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30815us (31240us total) trans 4294973039 rc 0/0 00000100:00100000:0.0:1789760756.743398:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5751 00000100:00100000:0.0:1789760756.743408:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735132806384 time 735177844677 state 0x2 ref 0 00000001:00080000:1.0:1789760756.744294:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973039 is committed 00000100:00100000:0.0:1789760756.752219:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760756.752329:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732224 00000100:00100000:2.0:1789760756.753607:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732224 00000100:00100000:2.0:1789760756.753673:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735132806384 time 735177844677 state 0x0 ref 1 00000100:00100000:2.0:1789760756.753688:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735178105093 time 735177844677 state 0x1 ref 1 00000100:00100000:2.0:1789760756.755668:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735178105093 time 735220945104 state 0x0 ref 1 00000100:00100000:2.0:1789760756.755698:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515732224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760756.755768:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23441408, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:2.0:1789760756.755779:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.755795:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.755800:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.755806:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.755819:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.808336:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23441408 -> 23445504 00010000:00010000:2.0:1789760756.808356:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808369:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808379:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808386:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760756.808390:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808399:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808426:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808434:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1eee7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.808537:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515732224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52848us (56325us total) trans 4294973040 rc 0/0 00000100:00100000:2.0:1789760756.808548:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5752 00000100:00100000:2.0:1789760756.808557:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735178105093 time 735220945104 state 0x2 ref 0 00000001:00080000:3.0:1789760756.808623:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973040 is committed 00000100:00100000:2.0:1789760756.817550:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760756.817700:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732352 00000100:00100000:0.0:1789760756.817821:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732352 00000100:00100000:0.0:1789760756.817887:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735178105093 time 735220945104 state 0x0 ref 1 00000100:00100000:0.0:1789760756.817902:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735221205520 time 735220945104 state 0x1 ref 1 00000100:00100000:0.0:1789760756.817978:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735221205520 time 735283265865 state 0x0 ref 1 00000100:00100000:0.0:1789760756.817991:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515732352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760756.818040:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23445504, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb807 00010000:00010000:0.0:1789760756.818050:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.818070:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.818075:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.818083:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.818099:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.862370:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973041 is committed 00002000:00010000:0.0:1789760756.862488:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23445504 -> 23449600 00002000:00010000:0.0:1789760756.862493:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760756 -> 1789760757 00002000:00010000:0.0:1789760756.862496:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760756 -> 1789760757 00010000:00010000:0.0:1789760756.862510:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.862524:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.862534:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.862542:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760756.862680:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.862692:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.862782:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.862864:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eeee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.863013:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515732352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45019us (45466us total) trans 4294973041 rc 0/0 00000100:00100000:0.0:1789760756.863024:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5753 00000100:00100000:0.0:1789760756.863035:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735221205520 time 735283265865 state 0x2 ref 0 00000100:00100000:0.0:1789760756.876743:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760756.878978:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732480 00000100:00100000:3.0:1789760756.878993:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732480 00000100:00100000:0.0:1789760756.879238:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735221205520 time 735283265865 state 0x0 ref 1 00000100:00100000:0.0:1789760756.879254:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735283526281 time 735283265865 state 0x1 ref 1 00000100:00100000:0.0:1789760756.879325:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735283526281 time 735344614036 state 0x0 ref 1 00000100:00100000:0.0:1789760756.879339:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515732480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760756.879390:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23449600, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:0.0:1789760756.879400:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.879416:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.879420:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.879427:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.879447:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.917838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973042 is committed 00002000:00010000:2.0:1789760756.918012:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23449600 -> 23453696 00010000:00010000:2.0:1789760756.918028:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.918040:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.918049:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.918056:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760756.918060:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.918073:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.918092:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.918100:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1eef5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.918195:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515732480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38854us (41457us total) trans 4294973042 rc 0/0 00000100:00100000:2.0:1789760756.918205:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5754 00000100:00100000:2.0:1789760756.918215:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735283526281 time 735344614036 state 0x2 ref 0 00000100:00100000:3.0:1789760756.921665:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760756.922109:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732608 00000100:00100000:0.0:1789760756.922120:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732608 00000100:00100000:0.0:1789760756.922177:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735283526281 time 735344614036 state 0x0 ref 1 00000100:00100000:0.0:1789760756.922192:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735344874452 time 735344614036 state 0x1 ref 1 00000100:00100000:0.0:1789760756.922225:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735344874452 time 735387514652 state 0x0 ref 1 00000100:00100000:0.0:1789760756.922239:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515732608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760756.922288:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23453696, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:0.0:1789760756.922297:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.922313:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.922317:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.922325:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.922339:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.968441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973043 is committed 00002000:00010000:1.0:1789760756.968516:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23453696 -> 23457792 00010000:00010000:1.0:1789760756.968533:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.968545:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.968553:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.968561:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760756.968566:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.968576:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.968598:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.968605:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1eefc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.968750:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515732608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46508us (47088us total) trans 4294973043 rc 0/0 00000100:00100000:1.0:1789760756.968761:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5755 00000100:00100000:1.0:1789760756.968770:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735344874452 time 735387514652 state 0x2 ref 0 00000100:00100000:0.0:1789760756.971908:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760756.972039:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732736 00000100:00100000:2.0:1789760756.972050:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732736 00000100:00100000:2.0:1789760756.972105:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735344874452 time 735387514652 state 0x0 ref 1 00000100:00100000:2.0:1789760756.972120:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735387775068 time 735387514652 state 0x1 ref 1 00000100:00100000:2.0:1789760756.972178:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735387775068 time 735437468465 state 0x0 ref 1 00000100:00100000:2.0:1789760756.972188:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515732736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760756.972233:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23457792, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760756.972242:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.972258:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.972262:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.972269:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.972282:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.020923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973044 is committed 00002000:00010000:0.0:1789760757.021151:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23457792 -> 23461888 00010000:00010000:0.0:1789760757.021170:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.021186:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.021195:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.021201:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760757.021206:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.021216:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.021237:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.021245:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ef03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.021384:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515732736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49192us (49482us total) trans 4294973044 rc 0/0 00000100:00100000:0.0:1789760757.021540:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5756 00000100:00100000:0.0:1789760757.021550:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735387775068 time 735437468465 state 0x2 ref 0 00000100:00100000:0.0:1789760757.025379:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760757.025455:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732864 00000100:00100000:3.0:1789760757.025465:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732864 00000100:00100000:3.0:1789760757.025521:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735387775068 time 735437468465 state 0x0 ref 1 00000100:00100000:3.0:1789760757.025537:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735437728881 time 735437468465 state 0x1 ref 1 00000100:00100000:3.0:1789760757.025566:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735437728881 time 735490855856 state 0x0 ref 1 00000100:00100000:3.0:1789760757.025579:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515732864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760757.025632:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23461888, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:3.0:1789760757.025642:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.025659:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760757.025663:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.025670:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.025683:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.056149:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760757.077911:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973045 is committed 00002000:00010000:3.0:1789760757.078647:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23461888 -> 23465984 00010000:00010000:3.0:1789760757.078664:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.078678:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.078689:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.078696:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760757.078701:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.078743:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.078772:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.078781:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.078878:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515732864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53283us (53490us total) trans 4294973045 rc 0/0 00000100:00100000:3.0:1789760757.078888:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5757 00000100:00100000:3.0:1789760757.078901:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735437728881 time 735490855856 state 0x2 ref 0 00000100:00100000:3.0:1789760757.094222:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.094632:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515732992 00000100:00100000:2.0:1789760757.094644:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515732992 00000100:00100000:2.0:1789760757.094697:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735437728881 time 735490855856 state 0x0 ref 1 00000100:00100000:2.0:1789760757.094744:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735491116272 time 735490855856 state 0x1 ref 1 00000100:00100000:2.0:1789760757.094772:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735491116272 time 735560062006 state 0x0 ref 1 00000100:00100000:2.0:1789760757.094784:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515732992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.094833:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23465984, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.094841:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.094855:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760757.094860:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.094876:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.094890:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.145337:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973046 is committed 00002000:00010000:2.0:1789760757.145490:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23465984 -> 23470080 00010000:00010000:2.0:1789760757.145505:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.145517:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.145525:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.145532:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.145537:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.145546:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.145564:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.145572:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.145666:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515732992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50879us (51447us total) trans 4294973046 rc 0/0 00000100:00100000:2.0:1789760757.145676:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5758 00000100:00100000:2.0:1789760757.145686:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735491116272 time 735560062006 state 0x2 ref 0 00000100:00100000:0.0:1789760757.149482:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760757.151735:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733120 00000100:00100000:3.0:1789760757.153717:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733120 00000100:00100000:3.0:1789760757.154243:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735491116272 time 735560062006 state 0x0 ref 1 00000100:00100000:3.0:1789760757.154264:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735560322422 time 735560062006 state 0x1 ref 1 00000100:00100000:3.0:1789760757.154378:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735560322422 time 735619665528 state 0x0 ref 1 00000100:00100000:3.0:1789760757.154398:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760757.154466:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23470080, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:3.0:1789760757.154476:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.154496:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760757.154501:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.154519:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.154537:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.206326:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973047 is committed 00002000:00010000:3.0:1789760757.206540:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23470080 -> 23474176 00010000:00010000:3.0:1789760757.206556:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.206569:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.206578:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.206587:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760757.206657:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.206668:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.206689:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.206697:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.206889:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52494us (57416us total) trans 4294973047 rc 0/0 00000100:00100000:3.0:1789760757.206900:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5759 00000100:00100000:3.0:1789760757.206909:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735560322422 time 735619665528 state 0x2 ref 0 00000100:00100000:0.0:1789760757.211666:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.211925:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733248 00000100:00100000:2.0:1789760757.211935:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733248 00000100:00100000:2.0:1789760757.211991:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735560322422 time 735619665528 state 0x0 ref 1 00000100:00100000:2.0:1789760757.212006:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735619925944 time 735619665528 state 0x1 ref 1 00000100:00100000:2.0:1789760757.212035:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735619925944 time 735677324689 state 0x0 ref 1 00000100:00100000:2.0:1789760757.212047:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515733248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.212097:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23474176, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.212105:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.212121:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760757.212125:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.212132:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.212151:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.246592:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544415360:0@lo:400:kworker.0 00000100:00100000:0.0:1789760757.246675:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760757.246744:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789760757.246751:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544414976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760757.246779:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760757.246882:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e34a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544415488:0@lo:400:kworker.0 00000100:00100000:0.0:1789760757.246973:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760757.247002:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8029c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544415232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247002:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415488 00000100:00100000:3.0:1789760757.247010:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415488 00000100:00100000:0.0:1789760757.247026:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760757.247044:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1789760757.247048:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544415104:0@lo:400:kworker.0 00000100:00100000:0.0:1789760757.247067:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760757.247080:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415232 00000100:00100000:3.0:1789760757.247083:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415232 00000100:00100000:0.0:1789760757.247089:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544415616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247100:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415104 00000100:00100000:3.0:1789760757.247103:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415104 00000100:00100000:0.0:1789760757.247109:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760757.247118:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1789760757.247126:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544415488:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760757.247132:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415360 00000100:00100000:0.0:1789760757.247135:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415360 00000100:00100000:0.0:1789760757.247159:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415616 00000100:00100000:0.0:1789760757.247162:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415616 00000100:00100000:0.0:1789760757.247181:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1789760757.247182:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544415488:12345-0@lo:400:kworker.0 Request processed in 57us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760757.247187:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544415360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247188:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1789760757.247203:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1789760757.247208:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544415232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247238:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805526a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544415232:12345-0@lo:400:kworker.0 Request processed in 30us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.247244:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1789760757.247256:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:3.0:1789760757.247261:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544415104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247286:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805524700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544415104:12345-0@lo:400:kworker.0 Request processed in 25us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.247291:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:3.0:1789760757.247320:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1789760757.247326:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544415616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247355:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544415616:12345-0@lo:400:kworker.0 Request processed in 30us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.247360:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1789760757.247395:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544415104:0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247405:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544415616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247413:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8029c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544415232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.247441:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e34a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544415488:0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.248674:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544414976 00000100:00100000:3.0:1789760757.248684:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544414976 00000100:00100000:3.0:1789760757.248747:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760757.248756:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae909352c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544414976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760757.248895:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae909352c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544414976:12345-0@lo:400:kworker.0 Request processed in 140us (2114us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.248913:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760757.248973:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544414976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760757.249553:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e37100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544415360:12345-0@lo:400:kworker.0 Request processed in 2364us (2877us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760757.249562:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00080000:0.0:1789760757.249590:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760757.249595:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760757.249598:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760757.250202:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544415360:0@lo:400:kworker.0 00000001:00080000:3.0:1789760757.253972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973048 is committed 00002000:00010000:2.0:1789760757.254026:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23474176 -> 23478272 00010000:00010000:2.0:1789760757.254043:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.254055:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.254064:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.254071:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.254076:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.254085:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.254104:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.254112:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.254235:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515733248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42184us (42573us total) trans 4294973048 rc 0/0 00000100:00100000:2.0:1789760757.254249:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5760 00000100:00100000:2.0:1789760757.254259:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735619925944 time 735677324689 state 0x2 ref 0 00000100:00100000:0.0:1789760757.256419:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.256509:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733376 00000100:00100000:1.0:1789760757.256519:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733376 00000100:00100000:1.0:1789760757.256568:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735619925944 time 735677324689 state 0x0 ref 1 00000100:00100000:1.0:1789760757.256584:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735677585105 time 735677324689 state 0x1 ref 1 00000100:00100000:1.0:1789760757.256687:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735677585105 time 735721974531 state 0x0 ref 1 00000100:00100000:1.0:1789760757.256727:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760757.256775:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23478272, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:1.0:1789760757.256783:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.256799:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760757.256803:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.256809:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.256823:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.300541:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973049 is committed 00002000:00010000:2.0:1789760757.300633:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23478272 -> 23482368 00010000:00010000:2.0:1789760757.300649:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300659:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300668:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300675:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.300679:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300688:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300727:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300736:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1ef26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.300838:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44130us (44422us total) trans 4294973049 rc 0/0 00000100:00100000:2.0:1789760757.300847:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5761 00000100:00100000:2.0:1789760757.300858:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735677585105 time 735721974531 state 0x2 ref 0 00000100:00100000:0.0:1789760757.305437:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.305916:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733504 00000100:00100000:2.0:1789760757.305926:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733504 00000100:00100000:2.0:1789760757.305978:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735677585105 time 735721974531 state 0x0 ref 1 00000100:00100000:2.0:1789760757.305993:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735722234947 time 735721974531 state 0x1 ref 1 00000100:00100000:2.0:1789760757.306024:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735722234947 time 735771313388 state 0x0 ref 1 00000100:00100000:2.0:1789760757.306037:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515733504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.306084:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23482368, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.306093:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.306108:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760757.306113:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.306119:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.306131:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.334654:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973050 is committed 00002000:00010000:2.0:1789760757.334760:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23482368 -> 23486464 00010000:00010000:2.0:1789760757.334776:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.334796:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.334805:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.334812:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.334817:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.334826:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.334846:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.334854:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.334955:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515733504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28915us (29522us total) trans 4294973050 rc 0/0 00000100:00100000:2.0:1789760757.334967:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5762 00000100:00100000:2.0:1789760757.334977:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735722234947 time 735771313388 state 0x2 ref 0 00000100:00100000:0.0:1789760757.339187:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760757.339277:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733632 00000100:00100000:3.0:1789760757.339297:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733632 00000100:00100000:3.0:1789760757.339457:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735722234947 time 735771313388 state 0x0 ref 1 00000100:00100000:3.0:1789760757.339480:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735771573804 time 735771313388 state 0x1 ref 1 00000100:00100000:3.0:1789760757.339573:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735771573804 time 735804860487 state 0x0 ref 1 00000100:00100000:3.0:1789760757.339592:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760757.339665:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23486464, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:3.0:1789760757.339676:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.339699:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760757.339704:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.339755:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.339773:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.384849:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973051 is committed 00002000:00010000:0.0:1789760757.385060:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23486464 -> 23490560 00010000:00010000:0.0:1789760757.385076:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.385090:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.385101:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.385109:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760757.385114:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.385124:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.385153:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.385162:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1ef34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.385261:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45668us (46079us total) trans 4294973051 rc 0/0 00000100:00100000:0.0:1789760757.385271:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5763 00000100:00100000:0.0:1789760757.385282:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735771573804 time 735804860487 state 0x2 ref 0 00000100:00100000:2.0:1789760757.404939:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.405014:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733760 00000100:00100000:1.0:1789760757.405024:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733760 00000100:00100000:1.0:1789760757.405086:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735771573804 time 735804860487 state 0x0 ref 1 00000100:00100000:1.0:1789760757.405102:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735805120903 time 735804860487 state 0x1 ref 1 00000100:00100000:1.0:1789760757.405130:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735805120903 time 735870419518 state 0x0 ref 1 00000100:00100000:1.0:1789760757.405141:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515733760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760757.405190:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23490560, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:1.0:1789760757.405199:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.405215:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760757.405220:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.405226:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.405239:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.444853:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973052 is committed 00002000:00010000:0.0:1789760757.444909:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23490560 -> 23494656 00010000:00010000:0.0:1789760757.444924:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444936:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444946:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444953:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760757.444958:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444972:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444992:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.445000:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.445095:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515733760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39950us (40162us total) trans 4294973052 rc 0/0 00000100:00100000:0.0:1789760757.445105:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5764 00000100:00100000:0.0:1789760757.445115:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735805120903 time 735870419518 state 0x2 ref 0 00000100:00100000:1.0:1789760757.449691:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760757.449741:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515733888 00000100:00100000:3.0:1789760757.449751:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515733888 00000100:00100000:3.0:1789760757.450006:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735805120903 time 735870419518 state 0x0 ref 1 00000100:00100000:3.0:1789760757.450022:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735870679934 time 735870419518 state 0x1 ref 1 00000100:00100000:3.0:1789760757.450057:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735870679934 time 735915345581 state 0x0 ref 1 00000100:00100000:3.0:1789760757.450070:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760757.450118:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23494656, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:3.0:1789760757.450126:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.450142:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760757.450147:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.450153:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.450167:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.495103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973053 is committed 00002000:00010000:3.0:1789760757.495664:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23494656 -> 23498752 00010000:00010000:3.0:1789760757.495681:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.495693:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.495702:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.495772:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760757.495777:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.495848:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.495876:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.495884:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.496014:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515733888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45941us (46323us total) trans 4294973053 rc 0/0 00000100:00100000:3.0:1789760757.496028:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5765 00000100:00100000:3.0:1789760757.496038:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735870679934 time 735915345581 state 0x2 ref 0 00000100:00100000:1.0:1789760757.499156:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.499948:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734016 00000100:00100000:2.0:1789760757.499960:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734016 00000100:00100000:2.0:1789760757.500018:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735870679934 time 735915345581 state 0x0 ref 1 00000100:00100000:2.0:1789760757.500033:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735915605997 time 735915345581 state 0x1 ref 1 00000100:00100000:2.0:1789760757.500065:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735915605997 time 735965354444 state 0x0 ref 1 00000100:00100000:2.0:1789760757.500077:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515734016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.500123:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23498752, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.500132:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.500148:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760757.500152:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.500158:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.500172:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.550293:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973054 is committed 00002000:00010000:0.0:1789760757.550584:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23498752 -> 23502848 00010000:00010000:0.0:1789760757.550600:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.550613:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.550622:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.550634:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760757.550638:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.550647:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.550667:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.550674:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ef49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.551017:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515734016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50935us (51865us total) trans 4294973054 rc 0/0 00000100:00100000:1.0:1789760757.551027:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5766 00000100:00100000:1.0:1789760757.551037:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735915605997 time 735965354444 state 0x2 ref 0 00000100:00100000:0.0:1789760757.556943:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.557080:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734144 00000100:00100000:2.0:1789760757.557091:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734144 00000100:00100000:2.0:1789760757.557147:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735915605997 time 735965354444 state 0x0 ref 1 00000100:00100000:2.0:1789760757.557164:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735965614860 time 735965354444 state 0x1 ref 1 00000100:00100000:2.0:1789760757.557228:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735965614860 time 736022516486 state 0x0 ref 1 00000100:00100000:2.0:1789760757.557241:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515734144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.557289:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23502848, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.557298:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.557314:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.557319:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.557325:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.557339:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.585354:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973055 is committed 00002000:00010000:2.0:1789760757.585417:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23502848 -> 23506944 00010000:00010000:2.0:1789760757.585431:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.585445:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.585455:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.585463:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.585468:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.585486:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.585516:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.585525:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1ef50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.585613:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515734144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28369us (28677us total) trans 4294973055 rc 0/0 00000100:00100000:2.0:1789760757.585623:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5767 00000100:00100000:2.0:1789760757.585633:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735965614860 time 736022516486 state 0x2 ref 0 00000100:00100000:0.0:1789760757.588918:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.589095:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734272 00000100:00100000:2.0:1789760757.589105:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734272 00000100:00100000:2.0:1789760757.589156:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 735965614860 time 736022516486 state 0x0 ref 1 00000100:00100000:2.0:1789760757.589171:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736022776902 time 736022516486 state 0x1 ref 1 00000100:00100000:2.0:1789760757.589201:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736022776902 time 736054490065 state 0x0 ref 1 00000100:00100000:2.0:1789760757.589213:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515734272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.589259:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23506944, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.589268:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.589282:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.589286:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.589292:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.589704:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760757.615114:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23506944 -> 23511040 00010000:00010000:2.0:1789760757.615135:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.615150:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.615166:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.615173:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.615178:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.615188:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.615205:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.615212:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1ef57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.615291:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515734272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26075us (26377us total) trans 4294973056 rc 0/0 00000100:00100000:2.0:1789760757.615301:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5768 00000100:00100000:2.0:1789760757.615311:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736022776902 time 736054490065 state 0x2 ref 0 00000001:00080000:2.0:1789760757.615349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973056 is committed 00000100:00100000:1.0:1789760757.618289:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760757.618394:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734400 00000100:00100000:0.0:1789760757.618403:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734400 00000100:00100000:0.0:1789760757.618612:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736022776902 time 736054490065 state 0x0 ref 1 00000100:00100000:0.0:1789760757.618628:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736054750481 time 736054490065 state 0x1 ref 1 00000100:00100000:0.0:1789760757.618662:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736054750481 time 736083951311 state 0x0 ref 1 00000100:00100000:0.0:1789760757.618675:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515734400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760757.618762:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23511040, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:0.0:1789760757.618772:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.618794:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.618799:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.618805:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.618819:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760757.644441:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23511040 -> 23515136 00010000:00010000:2.0:1789760757.644473:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.644496:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.644506:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.644514:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.644520:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.644530:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.644551:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.644559:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ef5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.644653:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515734400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25975us (26369us total) trans 4294973057 rc 0/0 00000100:00100000:2.0:1789760757.644664:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5769 00000100:00100000:2.0:1789760757.644674:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736054750481 time 736083951311 state 0x2 ref 0 00000001:00080000:2.0:1789760757.644767:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973057 is committed 00000100:00100000:0.0:1789760757.651059:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.651157:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734528 00000100:00100000:1.0:1789760757.651167:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734528 00000100:00100000:1.0:1789760757.651218:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736054750481 time 736083951311 state 0x0 ref 1 00000100:00100000:1.0:1789760757.651233:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736084211727 time 736083951311 state 0x1 ref 1 00000100:00100000:1.0:1789760757.651265:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736084211727 time 736116553727 state 0x0 ref 1 00000100:00100000:1.0:1789760757.651352:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515734528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760757.651405:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23515136, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:1.0:1789760757.651414:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.651441:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.651446:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.651453:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.651467:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.683502:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.683550:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.683578:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.683604:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734656 00000100:00100000:1.0:1789760757.683614:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734656 00000100:00100000:1.0:1789760757.683661:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 398 00000100:00100000:1.0:1789760757.683671:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515734656:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760757.683758:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515734656:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 88us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760757.683766:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 398 00000100:00100000:1.0:1789760757.683871:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734784 00000100:00100000:1.0:1789760757.683876:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734784 00000100:00100000:1.0:1789760757.683913:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515734912 00000100:00100000:1.0:1789760757.683916:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515734912 00000100:00100000:1.0:1789760757.683939:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 471 00000100:00100000:1.0:1789760757.683946:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515734784:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760757.683978:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c62a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515734784:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 32us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760757.683985:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 471 00000100:00100000:1.0:1789760757.683998:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 472 00000100:00100000:1.0:1789760757.684004:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515734912:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760757.684028:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c61f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515734912:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 25us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760757.684035:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 472 00000100:00080000:1.0:1789760757.684439:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760757.687598:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973058 is committed 00002000:00010000:2.0:1789760757.687673:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23515136 -> 23519232 00010000:00010000:2.0:1789760757.687698:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.687744:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.687753:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.687760:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.687765:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.687774:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.687805:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.687813:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1ef65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.687902:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515734528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36549us (36846us total) trans 4294973058 rc 0/0 00000100:00100000:2.0:1789760757.687913:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5770 00000100:00100000:2.0:1789760757.687922:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736084211727 time 736116553727 state 0x2 ref 0 00000100:00100000:1.0:1789760757.693139:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760757.693231:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735040 00000100:00100000:3.0:1789760757.693241:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735040 00000100:00100000:3.0:1789760757.693292:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736084211727 time 736116553727 state 0x0 ref 1 00000100:00100000:3.0:1789760757.693306:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736116814143 time 736116553727 state 0x1 ref 1 00000100:00100000:3.0:1789760757.693367:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736116814143 time 736158655040 state 0x0 ref 1 00000100:00100000:3.0:1789760757.693379:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515735040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760757.693425:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23519232, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:3.0:1789760757.693435:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.693451:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.693455:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.693462:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.693476:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.719560:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973059 is committed 00002000:00010000:0.0:1789760757.719809:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23519232 -> 23523328 00010000:00010000:0.0:1789760757.719824:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.719840:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.719849:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.719856:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760757.719873:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.719883:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.719903:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.719911:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1ef6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.720007:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515735040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26625us (26873us total) trans 4294973059 rc 0/0 00000100:00100000:0.0:1789760757.720018:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5771 00000100:00100000:0.0:1789760757.720167:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736116814143 time 736158655040 state 0x2 ref 0 00000100:00100000:0.0:1789760757.725883:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.726128:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735168 00000100:00100000:1.0:1789760757.726139:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735168 00000100:00100000:1.0:1789760757.726193:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736116814143 time 736158655040 state 0x0 ref 1 00000100:00100000:1.0:1789760757.726207:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736158915456 time 736158655040 state 0x1 ref 1 00000100:00100000:1.0:1789760757.726230:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736158915456 time 736191519773 state 0x0 ref 1 00000100:00100000:1.0:1789760757.726243:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515735168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760757.726293:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23523328, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:1.0:1789760757.726302:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.726319:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.726324:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.726330:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.726354:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.763408:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23523328 -> 23527424 00010000:00010000:1.0:1789760757.763429:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.763442:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.763450:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.763457:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760757.763462:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.763470:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.763489:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.763497:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.763595:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515735168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37350us (37727us total) trans 4294973060 rc 0/0 00000100:00100000:1.0:1789760757.763606:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5772 00000100:00100000:1.0:1789760757.763616:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736158915456 time 736191519773 state 0x2 ref 0 00000001:00080000:0.0:1789760757.763691:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973060 is committed 00000100:00100000:2.0:1789760757.766880:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760757.768495:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735296 00000100:00100000:0.0:1789760757.768508:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735296 00000100:00100000:0.0:1789760757.768567:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736158915456 time 736191519773 state 0x0 ref 1 00000100:00100000:0.0:1789760757.768581:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736191780189 time 736191519773 state 0x1 ref 1 00000100:00100000:0.0:1789760757.768649:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736191780189 time 736233938209 state 0x0 ref 1 00000100:00100000:0.0:1789760757.768663:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515735296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760757.768738:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23527424, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:0.0:1789760757.768747:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.768769:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.768774:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.768780:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.768794:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.799345:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23527424 -> 23531520 00010000:00010000:1.0:1789760757.799364:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.799387:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.799396:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.799403:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760757.799408:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.799417:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.799435:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.799447:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1ef7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.799531:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515735296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30866us (32664us total) trans 4294973061 rc 0/0 00000100:00100000:1.0:1789760757.799541:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5773 00000100:00100000:1.0:1789760757.799550:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736191780189 time 736233938209 state 0x2 ref 0 00000001:00080000:3.0:1789760757.800145:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973061 is committed 00000100:00100000:2.0:1789760757.803082:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760757.803220:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735424 00000100:00100000:0.0:1789760757.803230:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735424 00000100:00100000:0.0:1789760757.803282:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736191780189 time 736233938209 state 0x0 ref 1 00000100:00100000:0.0:1789760757.803298:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736234198625 time 736233938209 state 0x1 ref 1 00000100:00100000:0.0:1789760757.803332:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736234198625 time 736268620513 state 0x0 ref 1 00000100:00100000:0.0:1789760757.803345:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515735424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760757.803488:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23531520, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:0.0:1789760757.803497:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.803570:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.803575:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.803581:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.803601:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.836095:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973062 is committed 00002000:00010000:1.0:1789760757.836326:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23531520 -> 23535616 00010000:00010000:1.0:1789760757.836342:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.836355:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.836364:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.836371:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760757.836376:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.836384:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.836410:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.836417:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1ef81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.836509:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515735424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33162us (33431us total) trans 4294973062 rc 0/0 00000100:00100000:1.0:1789760757.836519:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5774 00000100:00100000:1.0:1789760757.836528:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736234198625 time 736268620513 state 0x2 ref 0 00000100:00100000:0.0:1789760757.850054:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760757.850171:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735552 00000100:00100000:2.0:1789760757.850182:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735552 00000100:00100000:2.0:1789760757.850239:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736234198625 time 736268620513 state 0x0 ref 1 00000100:00100000:2.0:1789760757.850256:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736268880929 time 736268620513 state 0x1 ref 1 00000100:00100000:2.0:1789760757.850289:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736268880929 time 736315578178 state 0x0 ref 1 00000100:00100000:2.0:1789760757.850303:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515735552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760757.850353:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23535616, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb807 00010000:00010000:2.0:1789760757.850363:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.850380:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.850385:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.850392:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.850407:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.883840:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973063 is committed 00002000:00010000:3.0:1789760757.884070:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23535616 -> 23539712 00002000:00010000:3.0:1789760757.884075:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:3.0:1789760757.884078:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00010000:00010000:3.0:1789760757.884094:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.884107:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.884115:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.884123:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760757.884128:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.884137:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.884155:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.884163:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1ef88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.884673:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515735552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34366us (34622us total) trans 4294973063 rc 0/0 00000100:00100000:3.0:1789760757.884687:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5775 00000100:00100000:3.0:1789760757.884697:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736268880929 time 736315578178 state 0x2 ref 0 00000100:00100000:2.0:1789760757.890927:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.892668:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735680 00000100:00100000:1.0:1789760757.892681:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735680 00000100:00100000:1.0:1789760757.892757:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736268880929 time 736315578178 state 0x0 ref 1 00000100:00100000:1.0:1789760757.892783:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736315838594 time 736315578178 state 0x1 ref 1 00000100:00100000:1.0:1789760757.892806:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736315838594 time 736358096340 state 0x0 ref 1 00000100:00100000:1.0:1789760757.892818:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515735680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760757.892869:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23539712, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760757.892879:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.892894:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.892899:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.892905:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.892919:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.924392:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973064 is committed 00002000:00010000:2.0:1789760757.924603:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23539712 -> 23543808 00010000:00010000:2.0:1789760757.924618:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.924629:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.924638:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.924645:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760757.924650:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.924658:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.924677:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.924686:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1ef8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.924815:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515735680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31993us (33891us total) trans 4294973064 rc 0/0 00000100:00100000:2.0:1789760757.924826:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5776 00000100:00100000:2.0:1789760757.924836:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736315838594 time 736358096340 state 0x2 ref 0 00000100:00100000:2.0:1789760757.928468:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760757.928611:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735808 00000100:00100000:3.0:1789760757.928621:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735808 00000100:00100000:3.0:1789760757.928671:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736315838594 time 736358096340 state 0x0 ref 1 00000100:00100000:3.0:1789760757.928685:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736358356756 time 736358096340 state 0x1 ref 1 00000100:00100000:3.0:1789760757.928748:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736358356756 time 736394036623 state 0x0 ref 1 00000100:00100000:3.0:1789760757.928761:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515735808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760757.928814:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23543808, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:3.0:1789760757.928823:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.928838:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.928842:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.928848:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.928870:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.981130:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973065 is committed 00002000:00010000:3.0:1789760757.981688:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23543808 -> 23547904 00010000:00010000:3.0:1789760757.981723:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.981736:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.981745:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.981751:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760757.981756:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.981765:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.981869:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.981879:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1ef96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.982016:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515735808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53251us (53551us total) trans 4294973065 rc 0/0 00000100:00100000:3.0:1789760757.982028:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5777 00000100:00100000:3.0:1789760757.982038:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736358356756 time 736394036623 state 0x2 ref 0 00000100:00100000:0.0:1789760757.986232:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760757.986418:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515735936 00000100:00100000:1.0:1789760757.986428:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515735936 00000100:00100000:1.0:1789760757.986483:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736358356756 time 736394036623 state 0x0 ref 1 00000100:00100000:1.0:1789760757.986498:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736394297039 time 736394036623 state 0x1 ref 1 00000100:00100000:1.0:1789760757.986609:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736394297039 time 736451896587 state 0x0 ref 1 00000100:00100000:1.0:1789760757.986623:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515735936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760757.986672:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23547904, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760757.986681:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.986696:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.986701:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.986740:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.986755:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.036134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973066 is committed 00002000:00010000:1.0:1789760758.036170:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23547904 -> 23552000 00010000:00010000:1.0:1789760758.036189:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.036203:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.036213:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.036259:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.036266:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.036277:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.036297:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.036302:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ef9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.036569:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515735936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49803us (50202us total) trans 4294973066 rc 0/0 00000100:00100000:1.0:1789760758.036962:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5778 00000100:00100000:1.0:1789760758.036975:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736394297039 time 736451896587 state 0x2 ref 0 00000100:00100000:3.0:1789760758.042264:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.042366:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736064 00000100:00100000:0.0:1789760758.042377:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736064 00000100:00100000:0.0:1789760758.042483:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736394297039 time 736451896587 state 0x0 ref 1 00000100:00100000:0.0:1789760758.042498:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736452157003 time 736451896587 state 0x1 ref 1 00000100:00100000:0.0:1789760758.042530:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736452157003 time 736507818950 state 0x0 ref 1 00000100:00100000:0.0:1789760758.042543:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515736064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.042604:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23552000, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.042614:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.042631:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760758.042636:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.042643:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.042664:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760758.043912:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760758.068143:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973067 is committed 00002000:00010000:1.0:1789760758.068231:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23552000 -> 23556096 00010000:00010000:1.0:1789760758.068247:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.068259:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.068268:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.068276:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.068280:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.068289:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.068308:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.068316:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1efa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.068409:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515736064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25864us (26149us total) trans 4294973067 rc 0/0 00000100:00100000:1.0:1789760758.068419:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5779 00000100:00100000:1.0:1789760758.068428:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736452157003 time 736507818950 state 0x2 ref 0 00000100:00100000:1.0:1789760758.072480:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.072575:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736192 00000100:00100000:0.0:1789760758.072585:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736192 00000100:00100000:0.0:1789760758.072639:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736452157003 time 736507818950 state 0x0 ref 1 00000100:00100000:0.0:1789760758.072653:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736508079366 time 736507818950 state 0x1 ref 1 00000100:00100000:0.0:1789760758.072682:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736508079366 time 736537970935 state 0x0 ref 1 00000100:00100000:0.0:1789760758.072694:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515736192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.072776:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23556096, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.072795:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.072811:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760758.072816:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.072822:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.072838:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.084730:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760758.103179:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973068 is committed 00002000:00010000:0.0:1789760758.103284:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23556096 -> 23560192 00010000:00010000:0.0:1789760758.103299:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.103312:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.103321:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.103329:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.103334:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.103343:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.103362:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.103371:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.103462:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515736192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30764us (30988us total) trans 4294973068 rc 0/0 00000100:00100000:0.0:1789760758.103472:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5780 00000100:00100000:0.0:1789760758.103481:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736508079366 time 736537970935 state 0x2 ref 0 00000100:00100000:1.0:1789760758.106930:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760758.107151:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736320 00000100:00100000:3.0:1789760758.107161:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736320 00000100:00100000:3.0:1789760758.107214:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736508079366 time 736537970935 state 0x0 ref 1 00000100:00100000:3.0:1789760758.107230:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736538231351 time 736537970935 state 0x1 ref 1 00000100:00100000:3.0:1789760758.107258:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736538231351 time 736572547354 state 0x0 ref 1 00000100:00100000:3.0:1789760758.107271:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515736320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760758.107322:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23560192, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:3.0:1789760758.107331:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.107346:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760758.107350:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.107355:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.107368:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.148761:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973069 is committed 00002000:00010000:1.0:1789760758.148821:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23560192 -> 23564288 00010000:00010000:1.0:1789760758.148836:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.148848:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.148867:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.148876:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.148882:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.148891:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.148912:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.148920:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1efb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.149015:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515736320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41741us (42089us total) trans 4294973069 rc 0/0 00000100:00100000:1.0:1789760758.149026:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5781 00000100:00100000:1.0:1789760758.149035:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736538231351 time 736572547354 state 0x2 ref 0 00000100:00100000:3.0:1789760758.155205:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760758.155430:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736448 00000100:00100000:2.0:1789760758.155441:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736448 00000100:00100000:2.0:1789760758.155496:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736538231351 time 736572547354 state 0x0 ref 1 00000100:00100000:2.0:1789760758.155523:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736572807770 time 736572547354 state 0x1 ref 1 00000100:00100000:2.0:1789760758.155553:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736572807770 time 736620843045 state 0x0 ref 1 00000100:00100000:2.0:1789760758.155567:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515736448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760758.155614:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23564288, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:2.0:1789760758.155624:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.155640:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760758.155645:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.155651:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.155665:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.196653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973070 is committed 00002000:00010000:0.0:1789760758.196770:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23564288 -> 23568384 00010000:00010000:0.0:1789760758.196800:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.196814:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.196824:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.196831:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.196837:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.196846:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.196876:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.196884:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1efb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.196970:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515736448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41401us (41769us total) trans 4294973070 rc 0/0 00000100:00100000:0.0:1789760758.196984:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5782 00000100:00100000:0.0:1789760758.196995:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736572807770 time 736620843045 state 0x2 ref 0 00000100:00100000:2.0:1789760758.202696:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760758.202815:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736576 00000100:00100000:3.0:1789760758.202825:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736576 00000100:00100000:3.0:1789760758.202894:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736572807770 time 736620843045 state 0x0 ref 1 00000100:00100000:3.0:1789760758.202909:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736621103461 time 736620843045 state 0x1 ref 1 00000100:00100000:3.0:1789760758.202928:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736621103461 time 736668218460 state 0x0 ref 1 00000100:00100000:3.0:1789760758.202939:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515736576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760758.202988:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23568384, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:3.0:1789760758.202998:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.203012:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760758.203017:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.203023:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.203037:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760758.235588:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23568384 -> 23572480 00010000:00010000:0.0:1789760758.235607:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.235618:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.235626:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.235632:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.235636:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.235646:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.235665:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.235672:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1efc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.235805:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515736576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32862us (33113us total) trans 4294973071 rc 0/0 00000100:00100000:0.0:1789760758.235815:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5783 00000100:00100000:0.0:1789760758.235825:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736621103461 time 736668218460 state 0x2 ref 0 00000001:00080000:1.0:1789760758.236231:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973071 is committed 00000100:00100000:1.0:1789760758.247838:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.247969:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736704 00000100:00100000:0.0:1789760758.247980:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736704 00000100:00100000:0.0:1789760758.248035:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736621103461 time 736668218460 state 0x0 ref 1 00000100:00100000:0.0:1789760758.248050:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736668478876 time 736668218460 state 0x1 ref 1 00000100:00100000:0.0:1789760758.248088:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736668478876 time 736713372938 state 0x0 ref 1 00000100:00100000:0.0:1789760758.248100:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515736704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.248146:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23572480, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.248156:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.248171:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760758.248175:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.248181:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.248194:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.310178:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973072 is committed 00002000:00010000:0.0:1789760758.310611:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23572480 -> 23576576 00010000:00010000:0.0:1789760758.310632:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.310648:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.310657:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.310665:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.310671:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.310681:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.310704:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.310735:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1efc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.310872:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515736704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62758us (63027us total) trans 4294973072 rc 0/0 00000100:00100000:0.0:1789760758.310889:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5784 00000100:00100000:0.0:1789760758.310902:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736668478876 time 736713372938 state 0x2 ref 0 00000100:00100000:3.0:1789760758.316352:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760758.316558:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736832 00000100:00100000:1.0:1789760758.316573:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736832 00000100:00100000:1.0:1789760758.316649:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736668478876 time 736713372938 state 0x0 ref 1 00000100:00100000:1.0:1789760758.316668:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736713633354 time 736713372938 state 0x1 ref 1 00000100:00100000:1.0:1789760758.316704:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736713633354 time 736781992569 state 0x0 ref 1 00000100:00100000:1.0:1789760758.316776:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515736832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760758.317048:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23576576, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760758.317059:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.317087:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760758.317093:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.317101:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.317117:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.354757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973073 is committed 00002000:00010000:1.0:1789760758.355035:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23576576 -> 23580672 00010000:00010000:1.0:1789760758.355056:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.355071:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.355081:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.355103:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.355108:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.355118:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.355144:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.355153:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.355277:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515736832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38496us (38932us total) trans 4294973073 rc 0/0 00000100:00100000:1.0:1789760758.355289:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5785 00000100:00100000:1.0:1789760758.355301:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736713633354 time 736781992569 state 0x2 ref 0 00000100:00100000:1.0:1789760758.358373:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760758.358540:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515736960 00000100:00100000:2.0:1789760758.358553:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515736960 00000100:00100000:2.0:1789760758.358624:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736713633354 time 736781992569 state 0x0 ref 1 00000100:00100000:2.0:1789760758.358645:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736782252985 time 736781992569 state 0x1 ref 1 00000100:00100000:2.0:1789760758.358746:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736782252985 time 736824034334 state 0x0 ref 1 00000100:00100000:2.0:1789760758.358762:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515736960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760758.358831:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23580672, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:2.0:1789760758.358843:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.358873:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760758.358879:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.358886:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.358903:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.400640:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973074 is committed 00002000:00010000:1.0:1789760758.400692:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23580672 -> 23584768 00010000:00010000:1.0:1789760758.400750:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.400767:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.400811:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.400820:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.400826:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.400837:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.400881:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.400955:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1efd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.401072:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515736960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42306us (42706us total) trans 4294973074 rc 0/0 00000100:00100000:1.0:1789760758.401815:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5786 00000100:00100000:1.0:1789760758.401829:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736782252985 time 736824034334 state 0x2 ref 0 00000100:00100000:0.0:1789760758.408106:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760758.414028:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737088 00000100:00100000:3.0:1789760758.414051:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737088 00000100:00100000:3.0:1789760758.414141:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736782252985 time 736824034334 state 0x0 ref 1 00000100:00100000:3.0:1789760758.414162:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736824294750 time 736824034334 state 0x1 ref 1 00000100:00100000:3.0:1789760758.414194:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736824294750 time 736879482260 state 0x0 ref 1 00000100:00100000:3.0:1789760758.414213:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515737088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760758.414283:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23584768, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:3.0:1789760758.414294:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.414316:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760758.414321:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.414330:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.414345:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.447679:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973075 is committed 00002000:00010000:0.0:1789760758.447935:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23584768 -> 23588864 00010000:00010000:0.0:1789760758.447956:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.447970:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.447980:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.447988:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.447995:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.448005:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.448031:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.448040:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1efdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.448169:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515737088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33952us (40071us total) trans 4294973075 rc 0/0 00000100:00100000:0.0:1789760758.448182:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5787 00000100:00100000:0.0:1789760758.448195:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736824294750 time 736879482260 state 0x2 ref 0 00000100:00100000:3.0:1789760758.452053:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.452236:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737216 00000100:00100000:0.0:1789760758.452249:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737216 00000100:00100000:0.0:1789760758.452322:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736824294750 time 736879482260 state 0x0 ref 1 00000100:00100000:0.0:1789760758.452339:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736879742676 time 736879482260 state 0x1 ref 1 00000100:00100000:0.0:1789760758.452635:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736879742676 time 736917922309 state 0x0 ref 1 00000100:00100000:0.0:1789760758.452800:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.452877:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23588864, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.452889:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.452910:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760758.452915:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.452923:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.452940:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.492820:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973076 is committed 00002000:00010000:0.0:1789760758.493261:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23588864 -> 23592960 00010000:00010000:0.0:1789760758.493282:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.493297:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.493308:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.493317:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.493324:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.493334:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.493358:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.493367:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1efe3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.493585:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40929us (41543us total) trans 4294973076 rc 0/0 00000100:00100000:0.0:1789760758.493602:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5788 00000100:00100000:0.0:1789760758.493616:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736879742676 time 736917922309 state 0x2 ref 0 00000100:00100000:2.0:1789760758.497226:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.497336:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737344 00000100:00100000:0.0:1789760758.497350:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737344 00000100:00100000:0.0:1789760758.497422:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736879742676 time 736917922309 state 0x0 ref 1 00000100:00100000:0.0:1789760758.497440:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736918182725 time 736917922309 state 0x1 ref 1 00000100:00100000:0.0:1789760758.497589:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736918182725 time 736962875602 state 0x0 ref 1 00000100:00100000:0.0:1789760758.497604:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515737344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.497679:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23592960, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.497689:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.497747:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760758.497754:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.497761:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.497787:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.529399:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973077 is committed 00002000:00010000:0.0:1789760758.529566:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23592960 -> 23597056 00010000:00010000:0.0:1789760758.529588:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.529604:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.529614:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.529623:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.529629:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.529640:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.529669:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.529677:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1efea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.530006:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515737344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32396us (32788us total) trans 4294973077 rc 0/0 00000100:00100000:0.0:1789760758.530020:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5789 00000100:00100000:0.0:1789760758.530034:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736918182725 time 736962875602 state 0x2 ref 0 00000100:00100000:1.0:1789760758.534152:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.534305:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737472 00000100:00100000:0.0:1789760758.534321:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737472 00000100:00100000:0.0:1789760758.534396:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736918182725 time 736962875602 state 0x0 ref 1 00000100:00100000:0.0:1789760758.534416:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736963136018 time 736962875602 state 0x1 ref 1 00000100:00100000:0.0:1789760758.534520:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736963136018 time 736999807208 state 0x0 ref 1 00000100:00100000:0.0:1789760758.534536:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515737472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.534610:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23597056, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.534621:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.534641:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760758.534646:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.534653:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.534669:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.579801:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973078 is committed 00002000:00010000:2.0:1789760758.579936:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23597056 -> 23601152 00010000:00010000:2.0:1789760758.579957:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.579973:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.579990:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.579998:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760758.580004:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.580014:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.580042:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.580051:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1eff1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.580215:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515737472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45675us (46070us total) trans 4294973078 rc 0/0 00000100:00100000:2.0:1789760758.580228:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5790 00000100:00100000:2.0:1789760758.580240:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736963136018 time 736999807208 state 0x2 ref 0 00000100:00100000:0.0:1789760758.583400:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760758.583483:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737600 00000100:00100000:1.0:1789760758.583559:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737600 00000100:00100000:1.0:1789760758.583629:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 736963136018 time 736999807208 state 0x0 ref 1 00000100:00100000:1.0:1789760758.583648:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737000067624 time 736999807208 state 0x1 ref 1 00000100:00100000:1.0:1789760758.583681:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737000067624 time 737048970423 state 0x0 ref 1 00000100:00100000:1.0:1789760758.583696:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760758.583794:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23601152, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760758.583806:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.583832:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.583838:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.583845:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.583872:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760758.618597:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23601152 -> 23605248 00010000:00010000:0.0:1789760758.618624:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.618641:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.618657:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.618665:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.618671:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.618681:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.618757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973079 is committed 00010000:00010000:0.0:1789760758.618782:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.618792:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1eff8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.618970:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35269us (35577us total) trans 4294973079 rc 0/0 00000100:00100000:0.0:1789760758.618983:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5791 00000100:00100000:0.0:1789760758.618997:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737000067624 time 737048970423 state 0x2 ref 0 00000100:00100000:0.0:1789760758.622439:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760758.622749:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737728 00000100:00100000:1.0:1789760758.622762:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737728 00000100:00100000:1.0:1789760758.622843:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737000067624 time 737048970423 state 0x0 ref 1 00000100:00100000:1.0:1789760758.622871:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737049230839 time 737048970423 state 0x1 ref 1 00000100:00100000:1.0:1789760758.622901:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737049230839 time 737088190325 state 0x0 ref 1 00000100:00100000:1.0:1789760758.622914:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515737728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760758.622976:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23605248, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760758.622986:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.623007:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.623012:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.623020:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.623043:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.659883:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973080 is committed 00002000:00010000:1.0:1789760758.660046:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23605248 -> 23609344 00010000:00010000:1.0:1789760758.660067:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.660081:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.660092:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.660100:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.660106:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.660115:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.660138:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.660147:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1efff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.660316:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515737728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37396us (37888us total) trans 4294973080 rc 0/0 00000100:00100000:1.0:1789760758.660329:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5792 00000100:00100000:1.0:1789760758.660341:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737049230839 time 737088190325 state 0x2 ref 0 00000100:00100000:1.0:1789760758.663391:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760758.663471:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737856 00000100:00100000:2.0:1789760758.663485:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737856 00000100:00100000:2.0:1789760758.663614:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737049230839 time 737088190325 state 0x0 ref 1 00000100:00100000:2.0:1789760758.663635:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737088450741 time 737088190325 state 0x1 ref 1 00000100:00100000:2.0:1789760758.663737:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737088450741 time 737129025907 state 0x0 ref 1 00000100:00100000:2.0:1789760758.663752:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760758.663824:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23609344, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:2.0:1789760758.663835:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.663865:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760758.663870:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.663877:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.663894:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.697914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973081 is committed 00002000:00010000:2.0:1789760758.698057:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23609344 -> 23613440 00010000:00010000:2.0:1789760758.698077:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.698093:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.698103:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.698117:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760758.698122:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.698132:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.698156:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.698164:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f006 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.698287:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34530us (34903us total) trans 4294973081 rc 0/0 00000100:00100000:2.0:1789760758.698300:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5793 00000100:00100000:2.0:1789760758.698312:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737088450741 time 737129025907 state 0x2 ref 0 00000100:00100000:2.0:1789760758.701846:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.702019:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515737984 00000100:00100000:0.0:1789760758.702033:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515737984 00000100:00100000:0.0:1789760758.702110:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737088450741 time 737129025907 state 0x0 ref 1 00000100:00100000:0.0:1789760758.702129:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737129286323 time 737129025907 state 0x1 ref 1 00000100:00100000:0.0:1789760758.702170:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737129286323 time 737167459027 state 0x0 ref 1 00000100:00100000:0.0:1789760758.702185:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.702248:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23613440, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:0.0:1789760758.702259:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.702279:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.702284:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.702292:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.702309:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.754242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973082 is committed 00002000:00010000:0.0:1789760758.754382:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23613440 -> 23617536 00010000:00010000:0.0:1789760758.754404:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.754432:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.754441:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.754452:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760758.754458:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.754469:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.754503:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.754512:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f00d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.754638:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515737984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52448us (52798us total) trans 4294973082 rc 0/0 00000100:00100000:0.0:1789760758.754650:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5794 00000100:00100000:0.0:1789760758.754663:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737129286323 time 737167459027 state 0x2 ref 0 00000100:00100000:0.0:1789760758.758202:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760758.758344:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738112 00000100:00100000:3.0:1789760758.758358:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738112 00000100:00100000:3.0:1789760758.758449:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737129286323 time 737167459027 state 0x0 ref 1 00000100:00100000:3.0:1789760758.758471:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737167719443 time 737167459027 state 0x1 ref 1 00000100:00100000:1.0:1789760758.758648:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737167719443 time 737223933958 state 0x0 ref 1 00000100:00100000:1.0:1789760758.758668:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515738112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760758.758769:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23617536, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760758.758790:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.758810:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.758815:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.758822:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.758838:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.795798:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23617536 -> 23621632 00010000:00010000:1.0:1789760758.795827:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.795849:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.795869:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.795877:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.795883:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.795894:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.795918:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.795926:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f014 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.796055:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515738112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37384us (37860us total) trans 4294973083 rc 0/0 00000100:00100000:1.0:1789760758.796068:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5795 00000100:00100000:1.0:1789760758.796080:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737167719443 time 737223933958 state 0x2 ref 0 00000001:00080000:1.0:1789760758.796135:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973083 is committed 00000100:00100000:1.0:1789760758.800574:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.800669:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738240 00000100:00100000:0.0:1789760758.800681:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738240 00000100:00100000:0.0:1789760758.800792:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737167719443 time 737223933958 state 0x0 ref 1 00000100:00100000:0.0:1789760758.800814:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737224194374 time 737223933958 state 0x1 ref 1 00000100:00100000:2.0:1789760758.800985:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737224194374 time 737266270167 state 0x0 ref 1 00000100:00100000:2.0:1789760758.801001:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760758.801064:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23621632, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:2.0:1789760758.801075:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801094:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760758.801099:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.801107:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801122:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.838666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973084 is committed 00002000:00010000:3.0:1789760758.838910:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23621632 -> 23625728 00010000:00010000:3.0:1789760758.838934:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.838954:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.838963:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.838970:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760758.838976:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.838986:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.839011:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.839018:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f01b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.839224:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38218us (38657us total) trans 4294973084 rc 0/0 00000100:00100000:3.0:1789760758.839237:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5796 00000100:00100000:3.0:1789760758.839251:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737224194374 time 737266270167 state 0x2 ref 0 00000100:00100000:2.0:1789760758.841500:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760758.841573:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738368 00000100:00100000:1.0:1789760758.841585:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738368 00000100:00100000:1.0:1789760758.841641:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737224194374 time 737266270167 state 0x0 ref 1 00000100:00100000:1.0:1789760758.841657:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737266530583 time 737266270167 state 0x1 ref 1 00000100:00100000:1.0:1789760758.841690:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737266530583 time 737306978988 state 0x0 ref 1 00000100:00100000:1.0:1789760758.841704:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515738368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760758.841798:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23625728, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb807 00010000:00010000:1.0:1789760758.841807:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.841824:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.841829:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.841834:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.841846:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.868146:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973085 is committed 00002000:00010000:1.0:1789760758.868301:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23625728 -> 23629824 00002000:00010000:1.0:1789760758.868307:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760758 -> 1789760759 00002000:00010000:1.0:1789760758.868309:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760758 -> 1789760759 00010000:00010000:1.0:1789760758.868325:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.868341:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.868350:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.868359:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.868365:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.868376:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.868400:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.868408:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f022 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.868525:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515738368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26816us (27029us total) trans 4294973085 rc 0/0 00000100:00100000:1.0:1789760758.868536:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5797 00000100:00100000:1.0:1789760758.868547:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737266530583 time 737306978988 state 0x2 ref 0 00000100:00100000:1.0:1789760758.873582:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760758.873697:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738496 00000100:00100000:3.0:1789760758.874007:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738496 00000100:00100000:2.0:1789760758.874172:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737266530583 time 737306978988 state 0x0 ref 1 00000100:00100000:2.0:1789760758.874194:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737307239404 time 737306978988 state 0x1 ref 1 00000100:00100000:2.0:1789760758.874274:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737307239404 time 737339560937 state 0x0 ref 1 00000100:00100000:2.0:1789760758.874290:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760758.874357:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23629824, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:2.0:1789760758.874369:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.874390:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760758.874396:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.874403:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.874426:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.909554:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973086 is committed 00002000:00010000:2.0:1789760758.910032:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23629824 -> 23633920 00010000:00010000:2.0:1789760758.910051:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.910065:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.910084:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.910092:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760758.910098:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.910108:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.910132:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.910139:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f029 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.910254:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35961us (36682us total) trans 4294973086 rc 0/0 00000100:00100000:2.0:1789760758.910266:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5798 00000100:00100000:2.0:1789760758.910278:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737307239404 time 737339560937 state 0x2 ref 0 00000100:00100000:2.0:1789760758.916257:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.916366:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738624 00000100:00100000:0.0:1789760758.916379:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738624 00000100:00100000:0.0:1789760758.916454:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737307239404 time 737339560937 state 0x0 ref 1 00000100:00100000:0.0:1789760758.916473:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737339821353 time 737339560937 state 0x1 ref 1 00000100:00100000:0.0:1789760758.916554:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737339821353 time 737381842302 state 0x0 ref 1 00000100:00100000:0.0:1789760758.916568:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515738624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.916631:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23633920, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:0.0:1789760758.916642:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.916664:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.916669:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.916676:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.916693:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.949294:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973087 is committed 00002000:00010000:1.0:1789760758.949429:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23633920 -> 23638016 00010000:00010000:1.0:1789760758.949452:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.949466:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.949476:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.949484:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760758.949490:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.949564:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.949590:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.949599:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f030 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.949770:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515738624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33195us (33518us total) trans 4294973087 rc 0/0 00000100:00100000:1.0:1789760758.949910:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5799 00000100:00100000:1.0:1789760758.949923:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737339821353 time 737381842302 state 0x2 ref 0 00000100:00100000:2.0:1789760758.953654:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760758.953951:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738752 00000100:00100000:0.0:1789760758.953966:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738752 00000100:00100000:0.0:1789760758.954043:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737339821353 time 737381842302 state 0x0 ref 1 00000100:00100000:0.0:1789760758.954063:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737382102718 time 737381842302 state 0x1 ref 1 00000100:00100000:0.0:1789760758.954130:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737382102718 time 737419418102 state 0x0 ref 1 00000100:00100000:0.0:1789760758.954148:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760758.954211:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23638016, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:0.0:1789760758.954221:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.954240:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.954246:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.954253:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.954270:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.013720:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973088 is committed 00002000:00010000:3.0:1789760759.014079:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23638016 -> 23642112 00010000:00010000:3.0:1789760759.014103:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.014118:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.014129:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.014137:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760759.014142:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.014152:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.014175:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.014183:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f037 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.014300:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60148us (60657us total) trans 4294973088 rc 0/0 00000100:00100000:3.0:1789760759.014319:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5800 00000100:00100000:3.0:1789760759.014331:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737382102718 time 737419418102 state 0x2 ref 0 00000100:00100000:2.0:1789760759.018059:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.018221:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515738880 00000100:00100000:1.0:1789760759.018235:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515738880 00000100:00100000:1.0:1789760759.018306:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737382102718 time 737419418102 state 0x0 ref 1 00000100:00100000:1.0:1789760759.018325:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737419678518 time 737419418102 state 0x1 ref 1 00000100:00100000:1.0:1789760759.018370:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737419678518 time 737483658551 state 0x0 ref 1 00000100:00100000:1.0:1789760759.018385:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.018459:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23642112, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.018470:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.018490:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760759.018495:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.018502:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.018519:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.101925:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760759.103193:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973089 is committed 00002000:00010000:1.0:1789760759.103291:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23642112 -> 23646208 00010000:00010000:1.0:1789760759.103306:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.103318:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.103328:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.103335:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.103340:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.103350:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.103368:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.103376:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f03e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.103497:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515738880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 85110us (85445us total) trans 4294973089 rc 0/0 00000100:00100000:1.0:1789760759.103507:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5801 00000100:00100000:1.0:1789760759.103517:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737419678518 time 737483658551 state 0x2 ref 0 00000100:00100000:2.0:1789760759.110813:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760759.110961:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739008 00000100:00100000:3.0:1789760759.110972:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739008 00000100:00100000:3.0:1789760759.111025:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737419678518 time 737483658551 state 0x0 ref 1 00000100:00100000:3.0:1789760759.111038:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737483918967 time 737483658551 state 0x1 ref 1 00000100:00100000:3.0:1789760759.111069:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737483918967 time 737576358118 state 0x0 ref 1 00000100:00100000:3.0:1789760759.111080:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515739008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760759.111128:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23646208, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:3.0:1789760759.111136:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.111151:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760759.111155:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.111161:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.111174:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.168264:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973090 is committed 00002000:00010000:2.0:1789760759.168599:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23646208 -> 23650304 00010000:00010000:2.0:1789760759.168616:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168628:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168636:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168642:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.168647:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168661:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168680:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.168689:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f045 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.169245:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515739008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58161us (58435us total) trans 4294973090 rc 0/0 00000100:00100000:2.0:1789760759.169255:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5802 00000100:00100000:2.0:1789760759.169266:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737483918967 time 737576358118 state 0x2 ref 0 00000100:00100000:3.0:1789760759.172973:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.173148:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739136 00000100:00100000:1.0:1789760759.173161:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739136 00000100:00100000:1.0:1789760759.173240:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737483918967 time 737576358118 state 0x0 ref 1 00000100:00100000:1.0:1789760759.173259:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737576618534 time 737576358118 state 0x1 ref 1 00000100:00100000:1.0:1789760759.173329:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737576618534 time 737638616504 state 0x0 ref 1 00000100:00100000:1.0:1789760759.173345:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515739136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.173411:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23650304, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.173422:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.173442:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760759.173447:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.173454:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.173470:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.235225:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973091 is committed 00002000:00010000:3.0:1789760759.235291:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23650304 -> 23654400 00010000:00010000:3.0:1789760759.235306:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.235320:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.235329:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.235336:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760759.235351:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.235361:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.235386:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.235394:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f04c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.235498:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515739136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62152us (62786us total) trans 4294973091 rc 0/0 00000100:00100000:3.0:1789760759.235510:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5803 00000100:00100000:3.0:1789760759.235600:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737576618534 time 737638616504 state 0x2 ref 0 00000100:00100000:1.0:1789760759.239640:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760759.240177:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739264 00000100:00100000:2.0:1789760759.240189:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739264 00000100:00100000:2.0:1789760759.240247:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737576618534 time 737638616504 state 0x0 ref 1 00000100:00100000:2.0:1789760759.240264:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737638876920 time 737638616504 state 0x1 ref 1 00000100:00100000:2.0:1789760759.240462:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737638876920 time 737705749599 state 0x0 ref 1 00000100:00100000:2.0:1789760759.240477:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515739264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760759.240528:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23654400, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:2.0:1789760759.240539:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.240556:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760759.240560:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.240566:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.240580:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.274208:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23654400 -> 23658496 00010000:00010000:2.0:1789760759.274227:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.274242:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.274252:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.274260:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.274265:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.274274:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.274303:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.274313:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f053 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.274406:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515739264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33927us (34772us total) trans 4294973092 rc 0/0 00000100:00100000:2.0:1789760759.274417:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5804 00000100:00100000:2.0:1789760759.274426:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737638876920 time 737705749599 state 0x2 ref 0 00000001:00080000:3.0:1789760759.275373:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973092 is committed 00000100:00100000:2.0:1789760759.280509:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.280582:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739392 00000100:00100000:1.0:1789760759.280592:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739392 00000100:00100000:1.0:1789760759.280644:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737638876920 time 737705749599 state 0x0 ref 1 00000100:00100000:1.0:1789760759.280661:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737706010015 time 737705749599 state 0x1 ref 1 00000100:00100000:1.0:1789760759.280691:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737706010015 time 737745980753 state 0x0 ref 1 00000100:00100000:1.0:1789760759.280738:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515739392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.280804:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23658496, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.280813:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.280828:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760759.280833:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.280839:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.280861:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.294001:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760759.294024:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544415872:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760759.294094:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.294161:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760759.294174:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544415744:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760759.294207:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760759.294297:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415872 00000100:00100000:3.0:1789760759.294305:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415872 00000100:00100000:3.0:1789760759.294367:0:6710:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544415744 00000100:00100000:3.0:1789760759.294371:0:6710:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544415744 00000100:00100000:3.0:1789760759.294393:0:6710:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789760759.294402:0:6710:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544415872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760759.294424:0:6710:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3834642432 avail 3792699392 00000100:00100000:3.0:1789760759.294472:0:6710:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544415872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760759.294480:0:6710:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1789760759.294893:0:11703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:2.0:1789760759.294905:0:11703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544415744:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760759.294930:0:11703:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760759.294979:0:11703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544415744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (771us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760759.295049:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544415744:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1789760759.295120:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760759.295125:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760759.296038:0:11703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:2.0:1789760759.299104:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544415872:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760759.322329:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973093 is committed 00002000:00010000:1.0:1789760759.322417:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23658496 -> 23662592 00010000:00010000:1.0:1789760759.322433:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.322445:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.322455:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.322467:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.322473:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.322481:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.322557:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.322563:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f05a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.322911:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515739392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42203us (42408us total) trans 4294973093 rc 0/0 00000100:00100000:2.0:1789760759.322923:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5805 00000100:00100000:2.0:1789760759.322932:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737706010015 time 737745980753 state 0x2 ref 0 00000100:00100000:1.0:1789760759.328251:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760759.328332:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739520 00000100:00100000:3.0:1789760759.328343:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739520 00000100:00100000:3.0:1789760759.328395:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737706010015 time 737745980753 state 0x0 ref 1 00000100:00100000:3.0:1789760759.328410:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737746241169 time 737745980753 state 0x1 ref 1 00000100:00100000:3.0:1789760759.328442:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737746241169 time 737793731314 state 0x0 ref 1 00000100:00100000:3.0:1789760759.328455:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515739520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760759.328506:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23662592, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:3.0:1789760759.328517:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.328534:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760759.328538:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.328546:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.328560:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.370548:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23662592 -> 23666688 00010000:00010000:2.0:1789760759.370579:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.370600:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.370610:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.370617:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.370623:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.370632:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.370647:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973094 is committed 00010000:00010000:2.0:1789760759.370660:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.370671:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f061 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.370864:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515739520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42396us (42610us total) trans 4294973094 rc 0/0 00000100:00100000:2.0:1789760759.370879:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5806 00000100:00100000:2.0:1789760759.370892:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737746241169 time 737793731314 state 0x2 ref 0 00000100:00100000:1.0:1789760759.374538:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760759.374629:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739648 00000100:00100000:0.0:1789760759.374639:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739648 00000100:00100000:0.0:1789760759.374695:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737746241169 time 737793731314 state 0x0 ref 1 00000100:00100000:0.0:1789760759.374874:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737793991730 time 737793731314 state 0x1 ref 1 00000100:00100000:0.0:1789760759.374946:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737793991730 time 737840233976 state 0x0 ref 1 00000100:00100000:0.0:1789760759.374958:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515739648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760759.375003:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23666688, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:0.0:1789760759.375013:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.375027:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760759.375031:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.375121:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.375137:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.410704:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973095 is committed 00002000:00010000:1.0:1789760759.410935:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23666688 -> 23670784 00010000:00010000:1.0:1789760759.410950:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.410964:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.410973:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.410985:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.410991:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.411000:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.411019:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.411027:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f068 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.411155:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515739648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36194us (36620us total) trans 4294973095 rc 0/0 00000100:00100000:1.0:1789760759.411166:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5807 00000100:00100000:1.0:1789760759.411176:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737793991730 time 737840233976 state 0x2 ref 0 00000100:00100000:1.0:1789760759.415223:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760759.415355:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739776 00000100:00100000:2.0:1789760759.415365:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739776 00000100:00100000:2.0:1789760759.415414:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737793991730 time 737840233976 state 0x0 ref 1 00000100:00100000:2.0:1789760759.415430:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737840494392 time 737840233976 state 0x1 ref 1 00000100:00100000:2.0:1789760759.415509:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737840494392 time 737880788195 state 0x0 ref 1 00000100:00100000:2.0:1789760759.415523:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515739776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760759.415571:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23670784, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:2.0:1789760759.415580:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.415595:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760759.415600:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.415606:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.415620:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.449348:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973096 is committed 00002000:00010000:1.0:1789760759.449508:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23670784 -> 23674880 00010000:00010000:1.0:1789760759.449523:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.449547:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.449561:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.449568:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.449572:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.449580:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.449602:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.449611:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f06f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.449700:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515739776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34174us (34483us total) trans 4294973096 rc 0/0 00000100:00100000:1.0:1789760759.449748:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5808 00000100:00100000:1.0:1789760759.449758:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737840494392 time 737880788195 state 0x2 ref 0 00000100:00100000:3.0:1789760759.455196:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.455428:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515739904 00000100:00100000:1.0:1789760759.455438:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515739904 00000100:00100000:1.0:1789760759.455489:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737840494392 time 737880788195 state 0x0 ref 1 00000100:00100000:1.0:1789760759.455516:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737881048611 time 737880788195 state 0x1 ref 1 00000100:00100000:1.0:1789760759.455546:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737881048611 time 737920835021 state 0x0 ref 1 00000100:00100000:1.0:1789760759.455558:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515739904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.455605:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23674880, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.455614:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.455628:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760759.455633:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.455639:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.455653:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.491170:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23674880 -> 23678976 00010000:00010000:1.0:1789760759.491191:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.491205:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.491215:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.491222:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.491227:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.491236:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.491256:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.491263:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f076 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.491352:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515739904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35792us (36161us total) trans 4294973097 rc 0/0 00000100:00100000:1.0:1789760759.491364:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5809 00000100:00100000:1.0:1789760759.491374:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737881048611 time 737920835021 state 0x2 ref 0 00000001:00080000:0.0:1789760759.492034:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973097 is committed 00000100:00100000:2.0:1789760759.494585:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760759.494747:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740032 00000100:00100000:0.0:1789760759.494759:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740032 00000100:00100000:0.0:1789760759.494817:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737881048611 time 737920835021 state 0x0 ref 1 00000100:00100000:0.0:1789760759.494831:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737921095437 time 737920835021 state 0x1 ref 1 00000100:00100000:0.0:1789760759.494859:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737921095437 time 737960148663 state 0x0 ref 1 00000100:00100000:0.0:1789760759.494871:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515740032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760759.494929:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23678976, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:0.0:1789760759.494943:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.494960:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760759.494964:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.494970:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.494986:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.534052:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973098 is committed 00002000:00010000:0.0:1789760759.534119:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23678976 -> 23683072 00010000:00010000:0.0:1789760759.534134:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.534145:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.534153:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.534159:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760759.534164:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.534174:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.534193:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.534202:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f07d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.534286:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515740032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39412us (39705us total) trans 4294973098 rc 0/0 00000100:00100000:0.0:1789760759.534297:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5810 00000100:00100000:0.0:1789760759.534306:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737921095437 time 737960148663 state 0x2 ref 0 00000100:00100000:1.0:1789760759.539841:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760759.539973:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740160 00000100:00100000:0.0:1789760759.539984:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740160 00000100:00100000:0.0:1789760759.540038:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737921095437 time 737960148663 state 0x0 ref 1 00000100:00100000:0.0:1789760759.540053:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737960409079 time 737960148663 state 0x1 ref 1 00000100:00100000:0.0:1789760759.540082:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737960409079 time 738005371322 state 0x0 ref 1 00000100:00100000:0.0:1789760759.540095:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515740160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760759.540147:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23683072, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:0.0:1789760759.540156:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.540172:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.540177:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.540184:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.540199:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.577089:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973099 is committed 00002000:00010000:2.0:1789760759.577335:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23683072 -> 23687168 00010000:00010000:2.0:1789760759.577509:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.577529:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.577538:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.577545:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.577549:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.577558:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.577576:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.577584:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f084 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.577676:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515740160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37578us (37841us total) trans 4294973099 rc 0/0 00000100:00100000:2.0:1789760759.577686:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5811 00000100:00100000:2.0:1789760759.577696:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737960409079 time 738005371322 state 0x2 ref 0 00000100:00100000:0.0:1789760759.582896:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760759.586589:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740288 00000100:00100000:2.0:1789760759.586605:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740288 00000100:00100000:2.0:1789760759.586669:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 737960409079 time 738005371322 state 0x0 ref 1 00000100:00100000:2.0:1789760759.586683:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738005631738 time 738005371322 state 0x1 ref 1 00000100:00100000:2.0:1789760759.586728:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738005631738 time 738052016993 state 0x0 ref 1 00000100:00100000:2.0:1789760759.586741:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515740288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760759.586788:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23687168, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:2.0:1789760759.586796:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.586811:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.586815:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.586821:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.586834:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760759.636083:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23687168 -> 23691264 00010000:00010000:0.0:1789760759.636102:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.636115:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.636124:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.636131:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760759.636136:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.636145:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.636162:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.636168:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f08b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.636300:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515740288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49555us (53407us total) trans 4294973100 rc 0/0 00000100:00100000:0.0:1789760759.636312:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5812 00000100:00100000:0.0:1789760759.636322:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738005631738 time 738052016993 state 0x2 ref 0 00000001:00080000:1.0:1789760759.636325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973100 is committed 00000100:00100000:3.0:1789760759.648371:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.650316:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740416 00000100:00100000:1.0:1789760759.650330:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740416 00000100:00100000:1.0:1789760759.650386:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738005631738 time 738052016993 state 0x0 ref 1 00000100:00100000:1.0:1789760759.650401:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738052277409 time 738052016993 state 0x1 ref 1 00000100:00100000:1.0:1789760759.650438:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738052277409 time 738115727300 state 0x0 ref 1 00000100:00100000:1.0:1789760759.650452:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515740416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.650501:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23691264, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.650509:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.650523:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.650528:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.650534:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.650547:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.684268:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973101 is committed 00002000:00010000:1.0:1789760759.684373:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23691264 -> 23695360 00010000:00010000:1.0:1789760759.684387:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.684401:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.684409:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.684415:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.684420:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.684428:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.684446:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.684454:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f092 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.684542:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515740416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34087us (36175us total) trans 4294973101 rc 0/0 00000100:00100000:1.0:1789760759.684551:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5813 00000100:00100000:1.0:1789760759.684560:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738052277409 time 738115727300 state 0x2 ref 0 00000100:00100000:3.0:1789760759.693326:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.693828:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740544 00000100:00100000:1.0:1789760759.693840:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740544 00000100:00100000:1.0:1789760759.693981:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738052277409 time 738115727300 state 0x0 ref 1 00000100:00100000:1.0:1789760759.693999:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738115987716 time 738115727300 state 0x1 ref 1 00000100:00100000:1.0:1789760759.694064:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738115987716 time 738159352869 state 0x0 ref 1 00000100:00100000:1.0:1789760759.694077:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515740544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.694129:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23695360, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.694138:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.694153:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.694158:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.694164:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.694177:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.731459:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23695360 -> 23699456 00010000:00010000:1.0:1789760759.731480:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.731493:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.731503:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.731510:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760759.731515:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.731524:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.731543:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.731551:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f099 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.731636:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515740544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37555us (38315us total) trans 4294973102 rc 0/0 00000100:00100000:1.0:1789760759.731646:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5814 00000100:00100000:1.0:1789760759.731655:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738115987716 time 738159352869 state 0x2 ref 0 00000001:00080000:1.0:1789760759.731791:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973102 is committed 00000100:00100000:0.0:1789760759.736107:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760759.736202:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740672 00000100:00100000:1.0:1789760759.736212:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740672 00000100:00100000:1.0:1789760759.736268:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738115987716 time 738159352869 state 0x0 ref 1 00000100:00100000:1.0:1789760759.736283:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738159613285 time 738159352869 state 0x1 ref 1 00000100:00100000:1.0:1789760759.736303:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738159613285 time 738201592838 state 0x0 ref 1 00000100:00100000:1.0:1789760759.736315:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515740672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.736364:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23699456, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.736373:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.736388:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.736392:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.736399:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.736412:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.790132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973103 is committed 00002000:00010000:2.0:1789760759.790874:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23699456 -> 23703552 00010000:00010000:2.0:1789760759.790892:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.790904:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.790918:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.790990:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.790997:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.791008:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.791028:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.791037:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f0a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.791203:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515740672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54818us (55036us total) trans 4294973103 rc 0/0 00000100:00100000:2.0:1789760759.791214:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5815 00000100:00100000:2.0:1789760759.791223:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738159613285 time 738201592838 state 0x2 ref 0 00000100:00100000:3.0:1789760759.796296:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760759.796439:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740800 00000100:00100000:0.0:1789760759.796451:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740800 00000100:00100000:0.0:1789760759.796524:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738159613285 time 738201592838 state 0x0 ref 1 00000100:00100000:0.0:1789760759.796543:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738201853254 time 738201592838 state 0x1 ref 1 00000100:00100000:1.0:1789760759.796769:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738201853254 time 738262054183 state 0x0 ref 1 00000100:00100000:1.0:1789760759.796868:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515740800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760759.796924:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23703552, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:1.0:1789760759.796932:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.796949:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.796953:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.796959:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.796973:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.830367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973104 is committed 00002000:00010000:2.0:1789760759.831135:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23703552 -> 23707648 00010000:00010000:2.0:1789760759.831151:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.831170:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.831180:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.831188:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.831193:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.831202:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.831222:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.831231:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f0a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.831325:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515740800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34457us (35034us total) trans 4294973104 rc 0/0 00000100:00100000:2.0:1789760759.831335:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5816 00000100:00100000:2.0:1789760759.831345:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738201853254 time 738262054183 state 0x2 ref 0 00000100:00100000:3.0:1789760759.838085:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760759.838187:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515740928 00000100:00100000:2.0:1789760759.838200:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515740928 00000100:00100000:2.0:1789760759.838254:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738201853254 time 738262054183 state 0x0 ref 1 00000100:00100000:2.0:1789760759.838269:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738262314599 time 738262054183 state 0x1 ref 1 00000100:00100000:2.0:1789760759.838345:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738262314599 time 738303633648 state 0x0 ref 1 00000100:00100000:2.0:1789760759.838359:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515740928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760759.838404:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23707648, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb807 00010000:00010000:2.0:1789760759.838413:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.838427:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.838431:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.838437:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.838450:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.885607:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973105 is committed 00002000:00010000:2.0:1789760759.885743:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23707648 -> 23711744 00002000:00010000:2.0:1789760759.885748:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760759 -> 1789760760 00002000:00010000:2.0:1789760759.885751:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760759 -> 1789760760 00010000:00010000:2.0:1789760759.885868:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.885883:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.885891:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.885898:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.885904:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.885912:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.885932:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.885939:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.886033:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515740928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47673us (47957us total) trans 4294973105 rc 0/0 00000100:00100000:2.0:1789760759.886043:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5817 00000100:00100000:2.0:1789760759.886052:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738262314599 time 738303633648 state 0x2 ref 0 00000100:00100000:0.0:1789760759.888701:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760759.888988:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741056 00000100:00100000:2.0:1789760759.888998:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741056 00000100:00100000:2.0:1789760759.889048:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738262314599 time 738303633648 state 0x0 ref 1 00000100:00100000:2.0:1789760759.889063:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738303894064 time 738303633648 state 0x1 ref 1 00000100:00100000:2.0:1789760759.889131:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738303894064 time 738354419910 state 0x0 ref 1 00000100:00100000:2.0:1789760759.889143:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760759.889187:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23711744, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760759.889195:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.889211:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.889215:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.889221:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.889233:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.945570:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973106 is committed 00002000:00010000:2.0:1789760759.945896:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23711744 -> 23715840 00010000:00010000:2.0:1789760759.945911:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.945928:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.945937:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.945944:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.945949:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.945957:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.945975:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.945983:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.946073:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56928us (57377us total) trans 4294973106 rc 0/0 00000100:00100000:2.0:1789760759.946083:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5818 00000100:00100000:2.0:1789760759.946093:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738303894064 time 738354419910 state 0x2 ref 0 00000100:00100000:1.0:1789760759.948497:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760759.948611:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741184 00000100:00100000:0.0:1789760759.948621:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741184 00000100:00100000:0.0:1789760759.948674:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738303894064 time 738354419910 state 0x0 ref 1 00000100:00100000:0.0:1789760759.948688:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738354680326 time 738354419910 state 0x1 ref 1 00000100:00100000:0.0:1789760759.948789:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738354680326 time 738414077877 state 0x0 ref 1 00000100:00100000:0.0:1789760759.948803:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760759.948867:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23715840, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:0.0:1789760759.948876:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.948891:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.948896:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.948903:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.948916:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.993613:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973107 is committed 00002000:00010000:2.0:1789760759.993677:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23715840 -> 23719936 00010000:00010000:2.0:1789760759.993693:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.993729:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.993740:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.993756:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760759.993772:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.993782:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.993802:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.993811:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f0bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.994080:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45272us (45586us total) trans 4294973107 rc 0/0 00000100:00100000:3.0:1789760759.994091:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5819 00000100:00100000:3.0:1789760759.994101:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738354680326 time 738414077877 state 0x2 ref 0 00000100:00100000:3.0:1789760760.003435:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760760.003585:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741312 00000100:00100000:1.0:1789760760.003595:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741312 00000100:00100000:1.0:1789760760.003666:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738354680326 time 738414077877 state 0x0 ref 1 00000100:00100000:1.0:1789760760.003681:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738414338293 time 738414077877 state 0x1 ref 1 00000100:00080000:0.0:1789760760.003893:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760760.004110:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738414338293 time 738469397358 state 0x0 ref 1 00000100:00100000:1.0:1789760760.004127:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515741312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760760.004175:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23719936, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:1.0:1789760760.004184:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.004199:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760760.004203:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.004209:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.004222:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.032737:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973108 is committed 00002000:00010000:3.0:1789760760.032963:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23719936 -> 23724032 00010000:00010000:3.0:1789760760.032979:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.032991:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.033000:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.033007:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760760.033011:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.033021:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.033040:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.033047:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f0c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.033179:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515741312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29051us (29750us total) trans 4294973108 rc 0/0 00000100:00100000:3.0:1789760760.033189:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5820 00000100:00100000:3.0:1789760760.033199:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738414338293 time 738469397358 state 0x2 ref 0 00000100:00100000:1.0:1789760760.036994:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.037164:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741440 00000100:00100000:2.0:1789760760.037173:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741440 00000100:00100000:2.0:1789760760.037247:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738414338293 time 738469397358 state 0x0 ref 1 00000100:00100000:2.0:1789760760.037263:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738469657774 time 738469397358 state 0x1 ref 1 00000100:00100000:2.0:1789760760.037298:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738469657774 time 738502587151 state 0x0 ref 1 00000100:00100000:2.0:1789760760.037651:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515741440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.037702:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23724032, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.037747:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.037768:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.037772:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.037779:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.037793:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.073176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973109 is committed 00002000:00010000:2.0:1789760760.073276:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23724032 -> 23728128 00010000:00010000:2.0:1789760760.073291:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.073303:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.073313:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.073320:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.073325:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.073332:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.073350:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.073358:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.073443:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515741440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36131us (36456us total) trans 4294973109 rc 0/0 00000100:00100000:2.0:1789760760.073452:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5821 00000100:00100000:2.0:1789760760.073463:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738469657774 time 738502587151 state 0x2 ref 0 00000100:00100000:1.0:1789760760.076361:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.076658:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741568 00000100:00100000:2.0:1789760760.076667:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741568 00000100:00100000:2.0:1789760760.076862:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738469657774 time 738502587151 state 0x0 ref 1 00000100:00100000:2.0:1789760760.076876:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738502847567 time 738502587151 state 0x1 ref 1 00000100:00100000:2.0:1789760760.076943:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738502847567 time 738542231406 state 0x0 ref 1 00000100:00100000:2.0:1789760760.076956:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515741568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.077004:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23728128, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.077014:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.077030:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.077034:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.077041:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.077060:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.101151:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973110 is committed 00002000:00010000:2.0:1789760760.101258:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23728128 -> 23732224 00010000:00010000:2.0:1789760760.101273:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.101285:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.101294:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.101301:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.101306:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.101314:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.101332:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.101340:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.101431:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515741568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24472us (25074us total) trans 4294973110 rc 0/0 00000100:00100000:2.0:1789760760.101440:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5822 00000100:00100000:2.0:1789760760.101449:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738502847567 time 738542231406 state 0x2 ref 0 00000100:00100000:3.0:1789760760.104818:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.104919:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741696 00000100:00100000:2.0:1789760760.104928:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741696 00000100:00100000:2.0:1789760760.104979:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738502847567 time 738542231406 state 0x0 ref 1 00000100:00100000:2.0:1789760760.104995:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738542491822 time 738542231406 state 0x1 ref 1 00000100:00100000:2.0:1789760760.105025:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738542491822 time 738570314289 state 0x0 ref 1 00000100:00100000:2.0:1789760760.105038:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.105084:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23732224, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.105093:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.105109:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.105114:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.105120:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.105132:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760760.126617:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23732224 -> 23736320 00010000:00010000:2.0:1789760760.126638:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.126652:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.126662:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.126669:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.126675:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.126685:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.126721:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.126730:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f0d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.127003:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515741696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21960us (22186us total) trans 4294973111 rc 0/0 00000100:00100000:2.0:1789760760.127018:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5823 00000100:00100000:2.0:1789760760.127028:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738542491822 time 738570314289 state 0x2 ref 0 00000001:00080000:0.0:1789760760.127159:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973111 is committed 00010000:00010000:2.0:1789760760.127250:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760760.130075:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.130163:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741824 00000100:00100000:2.0:1789760760.130172:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741824 00000100:00100000:2.0:1789760760.130220:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738542491822 time 738570314289 state 0x0 ref 1 00000100:00100000:2.0:1789760760.130235:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738570574705 time 738570314289 state 0x1 ref 1 00000100:00100000:2.0:1789760760.130306:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738570574705 time 738595595180 state 0x0 ref 1 00000100:00100000:2.0:1789760760.130319:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515741824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.130366:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23736320, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.130375:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.130391:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.130396:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.130403:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.130418:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760760.153648:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23736320 -> 23740416 00010000:00010000:2.0:1789760760.153668:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.153680:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.153688:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.153695:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.153699:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.153736:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.153756:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.153772:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.153863:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515741824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23541us (23792us total) trans 4294973112 rc 0/0 00000100:00100000:2.0:1789760760.153874:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5824 00000100:00100000:2.0:1789760760.153884:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738570574705 time 738595595180 state 0x2 ref 0 00000001:00080000:1.0:1789760760.153973:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973112 is committed 00000100:00100000:3.0:1789760760.157793:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760760.157943:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515741952 00000100:00100000:0.0:1789760760.157952:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515741952 00000100:00100000:0.0:1789760760.158004:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738570574705 time 738595595180 state 0x0 ref 1 00000100:00100000:0.0:1789760760.158020:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738595855596 time 738595595180 state 0x1 ref 1 00000100:00100000:0.0:1789760760.158051:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738595855596 time 738623340932 state 0x0 ref 1 00000100:00100000:0.0:1789760760.158064:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760760.158113:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23740416, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:0.0:1789760760.158122:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.158139:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760760.158143:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.158150:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.158164:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.198381:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973113 is committed 00002000:00010000:1.0:1789760760.198505:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23740416 -> 23744512 00010000:00010000:1.0:1789760760.198522:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.198535:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.198544:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.198557:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760760.198562:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.198570:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.198591:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.198599:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f0e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.198696:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515741952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40629us (40908us total) trans 4294973113 rc 0/0 00000100:00100000:1.0:1789760760.198739:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5825 00000100:00100000:1.0:1789760760.198750:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738595855596 time 738623340932 state 0x2 ref 0 00000100:00100000:3.0:1789760760.206005:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760760.206184:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742080 00000100:00100000:1.0:1789760760.206196:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742080 00000100:00100000:1.0:1789760760.206250:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738595855596 time 738623340932 state 0x0 ref 1 00000100:00100000:1.0:1789760760.206267:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738623601348 time 738623340932 state 0x1 ref 1 00000100:00100000:1.0:1789760760.206302:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738623601348 time 738671591959 state 0x0 ref 1 00000100:00100000:1.0:1789760760.206318:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515742080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760760.206367:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23744512, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:1.0:1789760760.206378:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.206394:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760760.206399:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.206405:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.206419:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.246900:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23744512 -> 23748608 00010000:00010000:1.0:1789760760.246921:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246934:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246942:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246950:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760760.246955:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246964:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246985:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.246993:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f0ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.247083:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515742080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40761us (41083us total) trans 4294973114 rc 0/0 00000100:00100000:1.0:1789760760.247094:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5826 00000100:00100000:1.0:1789760760.247103:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738623601348 time 738671591959 state 0x2 ref 0 00000001:00080000:2.0:1789760760.247231:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973114 is committed 00000100:00100000:3.0:1789760760.251181:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760760.251282:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742208 00000100:00100000:1.0:1789760760.251291:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742208 00000100:00100000:1.0:1789760760.251343:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738623601348 time 738671591959 state 0x0 ref 1 00000100:00100000:1.0:1789760760.251358:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738671852375 time 738671591959 state 0x1 ref 1 00000100:00100000:1.0:1789760760.251389:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738671852375 time 738716677772 state 0x0 ref 1 00000100:00100000:1.0:1789760760.251402:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515742208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760760.251450:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23748608, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:1.0:1789760760.251459:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.251475:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760760.251479:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.251497:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.251511:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.284859:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973115 is committed 00002000:00010000:2.0:1789760760.285127:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23748608 -> 23752704 00010000:00010000:2.0:1789760760.285144:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.285158:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.285167:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.285175:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.285180:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.285190:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.285209:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.285218:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f0f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.285312:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515742208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33907us (34135us total) trans 4294973115 rc 0/0 00000100:00100000:2.0:1789760760.285326:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5827 00000100:00100000:2.0:1789760760.285335:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738671852375 time 738716677772 state 0x2 ref 0 00000100:00100000:3.0:1789760760.290078:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.290177:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742336 00000100:00100000:2.0:1789760760.290188:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742336 00000100:00100000:2.0:1789760760.290243:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738671852375 time 738716677772 state 0x0 ref 1 00000100:00100000:2.0:1789760760.290257:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738716938188 time 738716677772 state 0x1 ref 1 00000100:00100000:2.0:1789760760.290323:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738716938188 time 738755611036 state 0x0 ref 1 00000100:00100000:2.0:1789760760.290336:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515742336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.290382:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23752704, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.290390:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.290404:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.290409:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.290415:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.290427:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.315925:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973116 is committed 00002000:00010000:2.0:1789760760.316082:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23752704 -> 23756800 00010000:00010000:2.0:1789760760.316097:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.316111:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.316120:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.316127:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.316132:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.316140:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.316160:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.316168:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f0fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.316256:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515742336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25918us (26183us total) trans 4294973116 rc 0/0 00000100:00100000:2.0:1789760760.316266:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5828 00000100:00100000:2.0:1789760760.316286:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738716938188 time 738755611036 state 0x2 ref 0 00000100:00100000:3.0:1789760760.319937:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.320015:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742464 00000100:00100000:2.0:1789760760.320024:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742464 00000100:00100000:2.0:1789760760.320075:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738716938188 time 738755611036 state 0x0 ref 1 00000100:00100000:2.0:1789760760.320087:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738755871452 time 738755611036 state 0x1 ref 1 00000100:00100000:2.0:1789760760.320111:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738755871452 time 738785401539 state 0x0 ref 1 00000100:00100000:2.0:1789760760.320120:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515742464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.320158:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23756800, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.320165:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.320178:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.320181:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.320186:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.320196:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.348429:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973117 is committed 00002000:00010000:2.0:1789760760.348579:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23756800 -> 23760896 00010000:00010000:2.0:1789760760.348594:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.348606:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.348619:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.348626:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.348630:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.348639:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.348659:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.348666:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f102 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.349101:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515742464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28975us (29168us total) trans 4294973117 rc 0/0 00000100:00100000:2.0:1789760760.349113:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5829 00000100:00100000:2.0:1789760760.349124:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738755871452 time 738785401539 state 0x2 ref 0 00000100:00100000:3.0:1789760760.354177:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.354257:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742592 00000100:00100000:2.0:1789760760.354269:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742592 00000100:00100000:2.0:1789760760.354326:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738755871452 time 738785401539 state 0x0 ref 1 00000100:00100000:2.0:1789760760.354343:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738785661955 time 738785401539 state 0x1 ref 1 00000100:00100000:2.0:1789760760.354528:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738785661955 time 738819815916 state 0x0 ref 1 00000100:00100000:2.0:1789760760.354542:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515742592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.354591:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23760896, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.354600:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.354618:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.354623:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.354639:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.354654:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.392425:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23760896 -> 23764992 00010000:00010000:3.0:1789760760.392448:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.392462:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.392471:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.392478:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760760.392553:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.392565:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.392584:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.392598:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f109 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.392698:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515742592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38154us (38528us total) trans 4294973118 rc 0/0 00000100:00100000:3.0:1789760760.392748:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5830 00000100:00100000:3.0:1789760760.392758:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738785661955 time 738819815916 state 0x2 ref 0 00000001:00080000:1.0:1789760760.392923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973118 is committed 00000100:00100000:3.0:1789760760.396664:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760760.396889:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742720 00000100:00100000:1.0:1789760760.396899:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742720 00000100:00100000:1.0:1789760760.396952:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738785661955 time 738819815916 state 0x0 ref 1 00000100:00100000:1.0:1789760760.396968:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738820076332 time 738819815916 state 0x1 ref 1 00000100:00100000:1.0:1789760760.397001:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738820076332 time 738862289501 state 0x0 ref 1 00000100:00100000:1.0:1789760760.397014:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515742720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760760.397065:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23764992, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:1.0:1789760760.397073:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.397089:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760760.397093:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.397100:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.397113:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.469648:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973119 is committed 00002000:00010000:2.0:1789760760.469935:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23764992 -> 23769088 00010000:00010000:2.0:1789760760.469951:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.469965:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.469973:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.469981:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.469986:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.469995:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470014:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.470022:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f110 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.470117:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515742720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73101us (73458us total) trans 4294973119 rc 0/0 00000100:00100000:2.0:1789760760.470127:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5831 00000100:00100000:2.0:1789760760.470138:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738820076332 time 738862289501 state 0x2 ref 0 00000100:00100000:2.0:1789760760.475252:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760760.475462:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742848 00000100:00100000:1.0:1789760760.475472:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742848 00000100:00100000:1.0:1789760760.475526:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738820076332 time 738862289501 state 0x0 ref 1 00000100:00100000:1.0:1789760760.475541:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738862549917 time 738862289501 state 0x1 ref 1 00000100:00100000:1.0:1789760760.475571:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738862549917 time 738940860018 state 0x0 ref 1 00000100:00100000:1.0:1789760760.475583:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515742848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760760.475632:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23769088, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:1.0:1789760760.475641:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.475656:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760760.475661:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.475667:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.475681:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.510385:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973120 is committed 00002000:00010000:2.0:1789760760.510411:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23769088 -> 23773184 00010000:00010000:2.0:1789760760.510427:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.510440:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.510450:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.510464:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.510470:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.510478:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.510496:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.510504:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f117 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.510586:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515742848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35000us (35338us total) trans 4294973120 rc 0/0 00000100:00100000:2.0:1789760760.510595:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5832 00000100:00100000:2.0:1789760760.510605:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738862549917 time 738940860018 state 0x2 ref 0 00000100:00100000:1.0:1789760760.515570:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.517578:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515742976 00000100:00100000:2.0:1789760760.517594:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515742976 00000100:00100000:2.0:1789760760.517657:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738862549917 time 738940860018 state 0x0 ref 1 00000100:00100000:2.0:1789760760.517671:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738941120434 time 738940860018 state 0x1 ref 1 00000100:00100000:2.0:1789760760.517724:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738941120434 time 738982995543 state 0x0 ref 1 00000100:00100000:2.0:1789760760.517739:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515742976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.517790:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23773184, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.517800:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.517816:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760760.517820:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.517827:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.517852:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.542245:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973121 is committed 00002000:00010000:2.0:1789760760.542341:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23773184 -> 23777280 00010000:00010000:2.0:1789760760.542356:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.542370:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.542379:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.542390:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.542396:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.542406:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.542428:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.542436:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f11e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.542544:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515742976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24802us (26978us total) trans 4294973121 rc 0/0 00000100:00100000:2.0:1789760760.542555:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5833 00000100:00100000:2.0:1789760760.542564:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738941120434 time 738982995543 state 0x2 ref 0 00000100:00100000:3.0:1789760760.546059:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.546142:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743104 00000100:00100000:2.0:1789760760.546153:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743104 00000100:00100000:2.0:1789760760.546207:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738941120434 time 738982995543 state 0x0 ref 1 00000100:00100000:2.0:1789760760.546221:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738983255959 time 738982995543 state 0x1 ref 1 00000100:00100000:2.0:1789760760.546255:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738983255959 time 739011544375 state 0x0 ref 1 00000100:00100000:2.0:1789760760.546266:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.546313:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23777280, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.546321:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.546337:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.546342:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.546348:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.546363:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.594640:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973122 is committed 00002000:00010000:2.0:1789760760.594766:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23777280 -> 23781376 00010000:00010000:2.0:1789760760.594785:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.594798:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.594808:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.594815:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.594820:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.594828:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.594859:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.594869:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f125 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.594959:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48691us (48906us total) trans 4294973122 rc 0/0 00000100:00100000:2.0:1789760760.594969:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5834 00000100:00100000:2.0:1789760760.594980:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738983255959 time 739011544375 state 0x2 ref 0 00000100:00100000:3.0:1789760760.598667:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.598891:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743232 00000100:00100000:2.0:1789760760.598902:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743232 00000100:00100000:2.0:1789760760.598959:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 738983255959 time 739011544375 state 0x0 ref 1 00000100:00100000:2.0:1789760760.598974:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739011804791 time 739011544375 state 0x1 ref 1 00000100:00100000:2.0:1789760760.599014:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739011804791 time 739064302690 state 0x0 ref 1 00000100:00100000:2.0:1789760760.599038:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.599089:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23781376, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.599098:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.599114:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.599119:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.599126:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.599146:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.635779:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973123 is committed 00002000:00010000:2.0:1789760760.635965:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23781376 -> 23785472 00010000:00010000:2.0:1789760760.635981:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.635994:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.636004:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.636012:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.636017:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.636027:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.636048:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.636056:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f12c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.636158:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37127us (37496us total) trans 4294973123 rc 0/0 00000100:00100000:2.0:1789760760.636169:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5835 00000100:00100000:2.0:1789760760.636180:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739011804791 time 739064302690 state 0x2 ref 0 00000100:00100000:1.0:1789760760.640413:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760760.640553:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743360 00000100:00100000:3.0:1789760760.640563:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743360 00000100:00100000:3.0:1789760760.640620:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739011804791 time 739064302690 state 0x0 ref 1 00000100:00100000:3.0:1789760760.640636:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739064563106 time 739064302690 state 0x1 ref 1 00000100:00100000:3.0:1789760760.640695:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739064563106 time 739105983710 state 0x0 ref 1 00000100:00100000:3.0:1789760760.640725:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515743360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760760.640785:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23785472, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:3.0:1789760760.640795:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.640809:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.640814:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.640819:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.640833:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.673134:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973124 is committed 00002000:00010000:3.0:1789760760.673265:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23785472 -> 23789568 00010000:00010000:3.0:1789760760.673290:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.673411:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.673421:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.673429:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760760.673433:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.673442:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.673463:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.673476:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f133 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.673577:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515743360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32866us (33168us total) trans 4294973124 rc 0/0 00000100:00100000:3.0:1789760760.673587:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5836 00000100:00100000:3.0:1789760760.673598:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739064563106 time 739105983710 state 0x2 ref 0 00000100:00100000:2.0:1789760760.678827:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760760.679001:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743488 00000100:00100000:3.0:1789760760.679011:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743488 00000100:00100000:3.0:1789760760.679063:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739064563106 time 739105983710 state 0x0 ref 1 00000100:00100000:3.0:1789760760.679084:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739106244126 time 739105983710 state 0x1 ref 1 00000100:00100000:3.0:1789760760.679113:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739106244126 time 739144402136 state 0x0 ref 1 00000100:00100000:3.0:1789760760.679125:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760760.679171:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23789568, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:3.0:1789760760.679180:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.679195:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.679200:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.679206:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.679224:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.714735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973125 is committed 00002000:00010000:3.0:1789760760.714941:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23789568 -> 23793664 00010000:00010000:3.0:1789760760.714955:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.714969:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.714979:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.714988:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760760.714993:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.715002:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.715022:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.715030:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f13a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.715123:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35995us (36301us total) trans 4294973125 rc 0/0 00000100:00100000:3.0:1789760760.715133:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5837 00000100:00100000:3.0:1789760760.715143:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739106244126 time 739144402136 state 0x2 ref 0 00000100:00100000:0.0:1789760760.720131:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760760.720216:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743616 00000100:00100000:3.0:1789760760.720227:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743616 00000100:00100000:3.0:1789760760.720592:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739106244126 time 739144402136 state 0x0 ref 1 00000100:00100000:3.0:1789760760.720611:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739144662552 time 739144402136 state 0x1 ref 1 00000100:00100000:3.0:1789760760.720690:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739144662552 time 739185978324 state 0x0 ref 1 00000100:00100000:3.0:1789760760.720743:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515743616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760760.720958:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23793664, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:3.0:1789760760.720968:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.720985:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.720990:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.720996:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.721009:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.759685:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973126 is committed 00002000:00010000:0.0:1789760760.759821:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23793664 -> 23797760 00010000:00010000:0.0:1789760760.759837:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759857:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759867:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759874:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760760.759879:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759892:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759913:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759921:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f141 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.760011:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515743616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39304us (39884us total) trans 4294973126 rc 0/0 00000100:00100000:0.0:1789760760.760021:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5838 00000100:00100000:0.0:1789760760.760030:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739144662552 time 739185978324 state 0x2 ref 0 00000100:00100000:1.0:1789760760.764686:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.764999:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743744 00000100:00100000:2.0:1789760760.765010:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743744 00000100:00100000:2.0:1789760760.765250:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739144662552 time 739185978324 state 0x0 ref 1 00000100:00100000:2.0:1789760760.765266:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739186238740 time 739185978324 state 0x1 ref 1 00000100:00100000:2.0:1789760760.765404:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739186238740 time 739230691769 state 0x0 ref 1 00000100:00100000:2.0:1789760760.765418:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.765471:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23797760, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.765491:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.765508:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.765513:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.765518:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.765537:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.797615:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973127 is committed 00002000:00010000:2.0:1789760760.797918:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23797760 -> 23801856 00010000:00010000:2.0:1789760760.797934:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.797948:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.797958:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.797966:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.797972:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.797981:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.798000:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.798007:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f148 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.798140:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32719us (33458us total) trans 4294973127 rc 0/0 00000100:00100000:2.0:1789760760.798151:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5839 00000100:00100000:2.0:1789760760.798161:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739186238740 time 739230691769 state 0x2 ref 0 00000100:00100000:2.0:1789760760.802404:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760760.802551:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515743872 00000100:00100000:1.0:1789760760.802561:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515743872 00000100:00100000:1.0:1789760760.802616:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739186238740 time 739230691769 state 0x0 ref 1 00000100:00100000:1.0:1789760760.802632:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739230952185 time 739230691769 state 0x1 ref 1 00000100:00100000:1.0:1789760760.802663:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739230952185 time 739267952085 state 0x0 ref 1 00000100:00100000:1.0:1789760760.802677:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515743872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760760.802769:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23801856, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:1.0:1789760760.802778:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.802794:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760760.802798:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.802805:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.802819:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.826897:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23801856 -> 23805952 00010000:00010000:0.0:1789760760.826917:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.826929:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.826939:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.826952:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760760.826957:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.826967:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.826986:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.826993:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f14f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.828456:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515743872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25773us (26053us total) trans 4294973128 rc 0/0 00000100:00100000:0.0:1789760760.828468:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5840 00000100:00100000:0.0:1789760760.828479:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739230952185 time 739267952085 state 0x2 ref 0 00000001:00080000:0.0:1789760760.828525:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973128 is committed 00000100:00100000:0.0:1789760760.831532:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760760.831615:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744000 00000100:00100000:2.0:1789760760.831626:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744000 00000100:00100000:2.0:1789760760.831681:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739230952185 time 739267952085 state 0x0 ref 1 00000100:00100000:2.0:1789760760.831697:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739268212501 time 739267952085 state 0x1 ref 1 00000100:00100000:2.0:1789760760.831761:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739268212501 time 739297050075 state 0x0 ref 1 00000100:00100000:2.0:1789760760.831774:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760760.831822:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23805952, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb807 00010000:00010000:2.0:1789760760.831832:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.831873:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.831878:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.831884:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.831906:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.867309:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973129 is committed 00002000:00010000:2.0:1789760760.867533:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23805952 -> 23810048 00002000:00010000:2.0:1789760760.867538:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760760 -> 1789760761 00002000:00010000:2.0:1789760760.867540:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760760 -> 1789760761 00010000:00010000:2.0:1789760760.867553:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.867566:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.867574:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.867581:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760760.867586:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.867594:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.867613:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.867620:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f156 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.867789:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36012us (36259us total) trans 4294973129 rc 0/0 00000100:00100000:2.0:1789760760.867800:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5841 00000100:00100000:2.0:1789760760.867810:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739268212501 time 739297050075 state 0x2 ref 0 00000100:00100000:1.0:1789760760.872462:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760760.873365:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744128 00000100:00100000:3.0:1789760760.873376:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744128 00000100:00100000:3.0:1789760760.873431:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739268212501 time 739297050075 state 0x0 ref 1 00000100:00100000:3.0:1789760760.873447:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739297310491 time 739297050075 state 0x1 ref 1 00000100:00100000:3.0:1789760760.873479:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739297310491 time 739338767850 state 0x0 ref 1 00000100:00100000:3.0:1789760760.873501:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515744128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760760.873548:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23810048, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760760.873557:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.873571:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.873576:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.873582:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.873595:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.914434:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973130 is committed 00002000:00010000:3.0:1789760760.914539:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23810048 -> 23814144 00010000:00010000:3.0:1789760760.914556:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.914575:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.914584:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.914591:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760760.914596:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.914605:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.914625:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.914632:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f15d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.914810:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515744128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41307us (42354us total) trans 4294973130 rc 0/0 00000100:00100000:3.0:1789760760.914820:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5842 00000100:00100000:3.0:1789760760.914830:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739297310491 time 739338767850 state 0x2 ref 0 00000100:00100000:2.0:1789760760.920031:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760760.920176:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744256 00000100:00100000:3.0:1789760760.920187:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744256 00000100:00100000:3.0:1789760760.920316:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739297310491 time 739338767850 state 0x0 ref 1 00000100:00100000:3.0:1789760760.920332:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739339028266 time 739338767850 state 0x1 ref 1 00000100:00100000:3.0:1789760760.920416:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739339028266 time 739385704462 state 0x0 ref 1 00000100:00100000:3.0:1789760760.920430:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760760.920478:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23814144, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760760.920498:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.920515:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.920519:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.920526:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.920541:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.959229:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973131 is committed 00002000:00010000:0.0:1789760760.959394:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23814144 -> 23818240 00010000:00010000:0.0:1789760760.959410:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.959422:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.959433:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.959441:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760760.959446:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.959455:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.959474:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.959482:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f164 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.959619:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39187us (39592us total) trans 4294973131 rc 0/0 00000100:00100000:0.0:1789760760.959630:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5843 00000100:00100000:0.0:1789760760.959638:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739339028266 time 739385704462 state 0x2 ref 0 00000100:00100000:0.0:1789760760.965283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760760.965372:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744384 00000100:00100000:3.0:1789760760.965389:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744384 00000100:00100000:3.0:1789760760.965469:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739339028266 time 739385704462 state 0x0 ref 1 00000100:00100000:3.0:1789760760.966331:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739385964878 time 739385704462 state 0x1 ref 1 00000100:00100000:3.0:1789760760.966377:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739385964878 time 739431665196 state 0x0 ref 1 00000100:00100000:3.0:1789760760.966405:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515744384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760760.967162:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23818240, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760760.967177:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.967197:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.967201:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.967209:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.967227:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.020729:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23818240 -> 23822336 00010000:00010000:2.0:1789760761.020770:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.020786:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.020797:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.020806:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.020812:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.020823:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.020858:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.020868:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f16b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.021001:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515744384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54599us (55730us total) trans 4294973132 rc 0/0 00000100:00100000:2.0:1789760761.021014:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5844 00000100:00100000:2.0:1789760761.021026:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739385964878 time 739431665196 state 0x2 ref 0 00000001:00080000:3.0:1789760761.021111:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973132 is committed 00000100:00100000:1.0:1789760761.032193:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760761.034116:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744512 00000100:00100000:2.0:1789760761.034135:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744512 00000100:00100000:2.0:1789760761.034222:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739385964878 time 739431665196 state 0x0 ref 1 00000100:00100000:2.0:1789760761.034242:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739431925612 time 739431665196 state 0x1 ref 1 00000100:00100000:2.0:1789760761.034286:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739431925612 time 739499574104 state 0x0 ref 1 00000100:00100000:2.0:1789760761.034301:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760761.034371:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23822336, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:2.0:1789760761.034381:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.034402:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760761.034408:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.034416:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.034437:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.085208:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973133 is committed 00002000:00010000:2.0:1789760761.085334:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23822336 -> 23826432 00010000:00010000:2.0:1789760761.085355:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.085371:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.085380:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.085390:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.085396:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.085407:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.085433:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.085442:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f172 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.085613:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51309us (53427us total) trans 4294973133 rc 0/0 00000100:00100000:2.0:1789760761.085626:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5845 00000100:00100000:2.0:1789760761.085637:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739431925612 time 739499574104 state 0x2 ref 0 00000100:00100000:0.0:1789760761.088943:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760761.089136:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744640 00000100:00100000:1.0:1789760761.089150:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744640 00000100:00100000:1.0:1789760761.089222:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739431925612 time 739499574104 state 0x0 ref 1 00000100:00100000:1.0:1789760761.089242:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739499834520 time 739499574104 state 0x1 ref 1 00000100:00100000:1.0:1789760761.089276:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739499834520 time 739554565500 state 0x0 ref 1 00000100:00100000:1.0:1789760761.089289:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760761.089355:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23826432, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:1.0:1789760761.089367:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089387:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760761.089392:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.089399:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089415:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.116548:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23826432 -> 23830528 00010000:00010000:2.0:1789760761.116578:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.116594:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.116616:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.116624:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.116630:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.116639:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.116665:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.116680:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f179 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.116874:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515744640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27580us (27939us total) trans 4294973134 rc 0/0 00000100:00100000:2.0:1789760761.116888:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5846 00000100:00100000:2.0:1789760761.116900:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739499834520 time 739554565500 state 0x2 ref 0 00000001:00080000:3.0:1789760761.120629:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973134 is committed 00000100:00100000:0.0:1789760761.121989:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760761.122138:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744768 00000100:00100000:2.0:1789760761.122153:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744768 00000100:00100000:2.0:1789760761.122228:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739499834520 time 739554565500 state 0x0 ref 1 00000100:00100000:2.0:1789760761.122248:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739554825916 time 739554565500 state 0x1 ref 1 00000100:00100000:2.0:1789760761.122316:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739554825916 time 739587604080 state 0x0 ref 1 00000100:00100000:2.0:1789760761.122330:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760761.122395:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23830528, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:2.0:1789760761.122406:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.122538:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760761.122545:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.122554:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.122578:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.150393:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760761.173534:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23830528 -> 23834624 00010000:00010000:1.0:1789760761.173565:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.173588:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.173598:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.173610:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760761.173616:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.173626:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.173648:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973135 is committed 00010000:00010000:1.0:1789760761.173652:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.173659:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f180 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.173817:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51481us (51833us total) trans 4294973135 rc 0/0 00000100:00100000:1.0:1789760761.174189:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5847 00000100:00100000:1.0:1789760761.174202:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739554825916 time 739587604080 state 0x2 ref 0 00000100:00100000:1.0:1789760761.176889:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760761.177347:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515744896 00000100:00100000:3.0:1789760761.177362:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515744896 00000100:00100000:3.0:1789760761.177435:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739554825916 time 739587604080 state 0x0 ref 1 00000100:00100000:3.0:1789760761.177454:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739587864496 time 739587604080 state 0x1 ref 1 00000100:00100000:3.0:1789760761.178900:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739587864496 time 739644175247 state 0x0 ref 1 00000100:00100000:3.0:1789760761.178934:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760761.179011:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23834624, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760761.179023:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.179044:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760761.179049:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.179058:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.179076:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.233565:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973136 is committed 00002000:00010000:2.0:1789760761.233695:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23834624 -> 23838720 00010000:00010000:2.0:1789760761.233734:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.233748:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.233759:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.233768:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.233775:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.233785:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.233810:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.233819:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f187 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.233960:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515744896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55033us (57078us total) trans 4294973136 rc 0/0 00000100:00100000:2.0:1789760761.233972:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5848 00000100:00100000:2.0:1789760761.233985:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739587864496 time 739644175247 state 0x2 ref 0 00000100:00100000:2.0:1789760761.240377:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760761.240512:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745024 00000100:00100000:0.0:1789760761.240526:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745024 00000100:00100000:0.0:1789760761.240598:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739587864496 time 739644175247 state 0x0 ref 1 00000100:00100000:0.0:1789760761.240618:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739644435663 time 739644175247 state 0x1 ref 1 00000100:00100000:0.0:1789760761.240686:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739644435663 time 739705973747 state 0x0 ref 1 00000100:00100000:0.0:1789760761.240701:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515745024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760761.240802:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23838720, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:0.0:1789760761.240814:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.240843:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760761.240849:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.240857:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.240875:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.292994:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973137 is committed 00002000:00010000:0.0:1789760761.293074:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23838720 -> 23842816 00010000:00010000:0.0:1789760761.293096:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.293111:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.293128:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.293138:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760761.293144:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.293154:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.293181:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.293189:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f18e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.293369:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515745024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52662us (53061us total) trans 4294973137 rc 0/0 00000100:00100000:0.0:1789760761.293382:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5849 00000100:00100000:0.0:1789760761.293395:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739644435663 time 739705973747 state 0x2 ref 0 00000100:00100000:0.0:1789760761.305103:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760761.307411:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745152 00000100:00100000:2.0:1789760761.307431:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745152 00000100:00100000:2.0:1789760761.307522:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739644435663 time 739705973747 state 0x0 ref 1 00000100:00100000:2.0:1789760761.307543:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739706234163 time 739705973747 state 0x1 ref 1 00000100:00100000:2.0:1789760761.307759:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739706234163 time 739773045590 state 0x0 ref 1 00000100:00100000:2.0:1789760761.307776:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515745152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760761.307851:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23842816, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:2.0:1789760761.307862:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.307879:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760761.307884:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.307891:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.307913:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.364794:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973138 is committed 00002000:00010000:2.0:1789760761.365131:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23842816 -> 23846912 00010000:00010000:2.0:1789760761.365152:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.365168:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.365178:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.365186:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.365193:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.365204:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.365230:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.365239:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f195 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.365378:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515745152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57598us (60281us total) trans 4294973138 rc 0/0 00000100:00100000:2.0:1789760761.365391:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5850 00000100:00100000:2.0:1789760761.365404:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739706234163 time 739773045590 state 0x2 ref 0 00000100:00100000:2.0:1789760761.374062:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760761.374228:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745280 00000100:00100000:1.0:1789760761.374242:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745280 00000100:00100000:1.0:1789760761.374380:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739706234163 time 739773045590 state 0x0 ref 1 00000100:00100000:1.0:1789760761.374401:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739773306006 time 739773045590 state 0x1 ref 1 00000100:00100000:1.0:1789760761.374436:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739773306006 time 739839724761 state 0x0 ref 1 00000100:00100000:1.0:1789760761.374451:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515745280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760761.374970:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23846912, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:1.0:1789760761.374983:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.375004:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760761.375010:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.375019:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.375037:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.447448:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973139 is committed 00002000:00010000:2.0:1789760761.447641:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23846912 -> 23851008 00010000:00010000:2.0:1789760761.447661:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.447677:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.447687:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.447695:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.447701:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.447749:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.447907:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.447921:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f19c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.448080:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515745280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73625us (74058us total) trans 4294973139 rc 0/0 00000100:00100000:2.0:1789760761.448093:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5851 00000100:00100000:2.0:1789760761.448106:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739773306006 time 739839724761 state 0x2 ref 0 00000100:00100000:2.0:1789760761.461996:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760761.462674:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745408 00000100:00100000:3.0:1789760761.462690:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745408 00000100:00100000:3.0:1789760761.462806:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739773306006 time 739839724761 state 0x0 ref 1 00000100:00100000:3.0:1789760761.462826:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739839985177 time 739839724761 state 0x1 ref 1 00000100:00100000:3.0:1789760761.462868:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739839985177 time 739928157018 state 0x0 ref 1 00000100:00100000:3.0:1789760761.462884:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515745408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760761.462951:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23851008, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760761.462963:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.462984:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760761.462990:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.462999:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.463017:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.510604:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973140 is committed 00002000:00010000:3.0:1789760761.510774:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23851008 -> 23855104 00010000:00010000:3.0:1789760761.510796:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.510812:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.510822:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.510840:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760761.510846:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.510856:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.510881:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.510890:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.511141:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515745408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48253us (49152us total) trans 4294973140 rc 0/0 00000100:00100000:3.0:1789760761.511154:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5852 00000100:00100000:3.0:1789760761.511167:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739839985177 time 739928157018 state 0x2 ref 0 00000100:00100000:1.0:1789760761.517275:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760761.519018:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745536 00000100:00100000:2.0:1789760761.519038:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745536 00000100:00100000:2.0:1789760761.519124:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739839985177 time 739928157018 state 0x0 ref 1 00000100:00100000:2.0:1789760761.519142:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739928417434 time 739928157018 state 0x1 ref 1 00000100:00100000:2.0:1789760761.519231:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739928417434 time 739984519296 state 0x0 ref 1 00000100:00100000:2.0:1789760761.519247:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515745536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760761.519317:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23855104, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:2.0:1789760761.519329:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.519350:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760761.519356:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.519365:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.519382:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.593380:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23855104 -> 23859200 00010000:00010000:1.0:1789760761.593411:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.593446:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.593456:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.593464:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760761.593470:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.593488:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.593517:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.593526:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f1aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.593649:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515745536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74397us (76383us total) trans 4294973141 rc 0/0 00000100:00100000:1.0:1789760761.593661:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5853 00000100:00100000:1.0:1789760761.593675:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739928417434 time 739984519296 state 0x2 ref 0 00000001:00080000:3.0:1789760761.594458:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973141 is committed 00000100:00100000:2.0:1789760761.597406:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760761.597676:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745664 00000100:00100000:3.0:1789760761.597689:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745664 00000100:00100000:3.0:1789760761.598353:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739928417434 time 739984519296 state 0x0 ref 1 00000100:00100000:3.0:1789760761.598373:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739984779712 time 739984519296 state 0x1 ref 1 00000100:00100000:3.0:1789760761.598448:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739984779712 time 740063736299 state 0x0 ref 1 00000100:00100000:3.0:1789760761.598464:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515745664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760761.598541:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23859200, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760761.598551:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.598569:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.598574:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.598582:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.598600:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.639931:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973142 is committed 00002000:00010000:1.0:1789760761.640285:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23859200 -> 23863296 00010000:00010000:1.0:1789760761.640309:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.640326:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.640336:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.640344:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760761.640350:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.640359:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.640395:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.640405:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f1b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.640522:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515745664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42055us (43126us total) trans 4294973142 rc 0/0 00000100:00100000:1.0:1789760761.640535:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5854 00000100:00100000:1.0:1789760761.640547:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739984779712 time 740063736299 state 0x2 ref 0 00000100:00100000:2.0:1789760761.654046:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760761.654184:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745792 00000100:00100000:0.0:1789760761.654200:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745792 00000100:00100000:0.0:1789760761.654464:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 739984779712 time 740063736299 state 0x0 ref 1 00000100:00100000:0.0:1789760761.654488:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740063996715 time 740063736299 state 0x1 ref 1 00000100:00100000:0.0:1789760761.654527:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740063996715 time 740119815433 state 0x0 ref 1 00000100:00100000:0.0:1789760761.654542:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515745792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760761.654604:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23863296, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:0.0:1789760761.654615:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.654635:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760761.654640:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.654647:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.654665:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.712521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973143 is committed 00002000:00010000:2.0:1789760761.712665:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23863296 -> 23867392 00010000:00010000:2.0:1789760761.712680:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.712692:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.712701:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.712736:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.712741:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.712761:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.712781:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.712795:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f1b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.712891:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515745792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58348us (58858us total) trans 4294973143 rc 0/0 00000100:00100000:2.0:1789760761.712902:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5855 00000100:00100000:2.0:1789760761.712911:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740063996715 time 740119815433 state 0x2 ref 0 00000100:00100000:3.0:1789760761.721817:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760761.721922:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515745920 00000100:00100000:0.0:1789760761.721932:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515745920 00000100:00100000:0.0:1789760761.721986:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740063996715 time 740119815433 state 0x0 ref 1 00000100:00100000:0.0:1789760761.722002:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740120075849 time 740119815433 state 0x1 ref 1 00000100:00100000:0.0:1789760761.722033:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740120075849 time 740187321837 state 0x0 ref 1 00000100:00100000:0.0:1789760761.722045:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515745920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760761.722093:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23867392, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:0.0:1789760761.722102:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.722119:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760761.722124:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.722130:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.722145:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.781266:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973144 is committed 00002000:00010000:1.0:1789760761.781360:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23867392 -> 23871488 00010000:00010000:1.0:1789760761.781380:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.781393:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.781402:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.781409:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760761.781414:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.781423:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.781443:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.781452:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f1bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.781550:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515745920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59502us (59737us total) trans 4294973144 rc 0/0 00000100:00100000:1.0:1789760761.781568:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5856 00000100:00100000:1.0:1789760761.781578:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740120075849 time 740187321837 state 0x2 ref 0 00000100:00100000:1.0:1789760761.785234:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760761.786166:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746048 00000100:00100000:2.0:1789760761.786188:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746048 00000100:00100000:2.0:1789760761.786286:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740120075849 time 740187321837 state 0x0 ref 1 00000100:00100000:2.0:1789760761.786307:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740187582253 time 740187321837 state 0x1 ref 1 00000100:00100000:2.0:1789760761.786347:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740187582253 time 740251635572 state 0x0 ref 1 00000100:00100000:2.0:1789760761.786363:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515746048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760761.786448:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23871488, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:2.0:1789760761.786460:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.786482:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.786488:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.786496:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.786515:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.823172:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23871488 -> 23875584 00010000:00010000:2.0:1789760761.823191:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.823204:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.823214:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.823221:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.823226:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.823236:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.823254:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.823261:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.823364:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515746048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36999us (38134us total) trans 4294973145 rc 0/0 00000100:00100000:2.0:1789760761.823374:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5857 00000100:00100000:2.0:1789760761.823384:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740187582253 time 740251635572 state 0x2 ref 0 00000001:00080000:3.0:1789760761.824027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973145 is committed 00000100:00100000:2.0:1789760761.832011:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760761.832287:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746176 00000100:00100000:3.0:1789760761.832298:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746176 00000100:00100000:3.0:1789760761.832349:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740187582253 time 740251635572 state 0x0 ref 1 00000100:00100000:3.0:1789760761.832364:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740251895988 time 740251635572 state 0x1 ref 1 00000100:00100000:3.0:1789760761.832392:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740251895988 time 740297680842 state 0x0 ref 1 00000100:00100000:3.0:1789760761.832404:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515746176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760761.832452:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23875584, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb807 00010000:00010000:3.0:1789760761.832461:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832477:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.832481:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.832487:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832501:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.870212:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973146 is committed 00002000:00010000:3.0:1789760761.870315:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23875584 -> 23879680 00002000:00010000:3.0:1789760761.870320:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:3.0:1789760761.870322:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00010000:00010000:3.0:1789760761.870335:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.870348:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.870357:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.870364:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760761.870370:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.870379:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.870399:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.870407:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.870494:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515746176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38087us (38664us total) trans 4294973146 rc 0/0 00000100:00100000:3.0:1789760761.870505:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5858 00000100:00100000:3.0:1789760761.870516:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740251895988 time 740297680842 state 0x2 ref 0 00000100:00100000:0.0:1789760761.873093:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760761.873526:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746304 00000100:00100000:1.0:1789760761.873536:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746304 00000100:00100000:1.0:1789760761.873603:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740251895988 time 740297680842 state 0x0 ref 1 00000100:00100000:1.0:1789760761.873618:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740297941258 time 740297680842 state 0x1 ref 1 00000100:00100000:1.0:1789760761.873651:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740297941258 time 740338939516 state 0x0 ref 1 00000100:00100000:1.0:1789760761.873663:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515746304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760761.874053:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23879680, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:1.0:1789760761.874065:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.874080:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.874085:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.874092:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.874107:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.907101:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973147 is committed 00002000:00010000:2.0:1789760761.907133:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23879680 -> 23883776 00010000:00010000:2.0:1789760761.907148:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.907162:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.907171:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.907178:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.907183:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.907192:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.907211:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.907220:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f1d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.907355:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515746304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33689us (34267us total) trans 4294973147 rc 0/0 00000100:00100000:2.0:1789760761.907365:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5859 00000100:00100000:2.0:1789760761.907376:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740297941258 time 740338939516 state 0x2 ref 0 00000100:00100000:1.0:1789760761.911704:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760761.917178:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746432 00000100:00100000:3.0:1789760761.917194:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746432 00000100:00100000:2.0:1789760761.917304:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740297941258 time 740338939516 state 0x0 ref 1 00000100:00100000:2.0:1789760761.917322:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740339199932 time 740338939516 state 0x1 ref 1 00000100:00100000:2.0:1789760761.917360:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740339199932 time 740382649252 state 0x0 ref 1 00000100:00100000:2.0:1789760761.917373:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515746432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760761.917425:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23883776, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760761.917434:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.917450:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760761.917455:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.917461:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.917475:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.949530:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23883776 -> 23887872 00010000:00010000:2.0:1789760761.949552:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.949565:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.949574:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.949581:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.949586:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.949595:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.949615:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.949623:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.949742:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515746432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32367us (38043us total) trans 4294973148 rc 0/0 00000100:00100000:2.0:1789760761.949768:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5860 00000100:00100000:2.0:1789760761.949779:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740339199932 time 740382649252 state 0x2 ref 0 00000001:00080000:0.0:1789760761.950205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973148 is committed 00000100:00100000:1.0:1789760761.955576:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760761.955744:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746560 00000100:00100000:2.0:1789760761.955763:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746560 00000100:00100000:2.0:1789760761.955819:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740339199932 time 740382649252 state 0x0 ref 1 00000100:00100000:2.0:1789760761.955847:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740382909668 time 740382649252 state 0x1 ref 1 00000100:00100000:3.0:1789760761.955968:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740382909668 time 740421255337 state 0x0 ref 1 00000100:00100000:3.0:1789760761.955982:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515746560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760761.956030:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23887872, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:3.0:1789760761.956039:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.956057:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.956062:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.956068:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.956082:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760761.990151:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23887872 -> 23891968 00010000:00010000:2.0:1789760761.990173:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.990186:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.990195:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.990202:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760761.990207:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.990216:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.990234:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.990243:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f1e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.990324:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515746560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34340us (34753us total) trans 4294973149 rc 0/0 00000100:00100000:2.0:1789760761.990335:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5861 00000100:00100000:2.0:1789760761.990344:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740382909668 time 740421255337 state 0x2 ref 0 00000001:00080000:0.0:1789760761.990416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973149 is committed 00000100:00100000:1.0:1789760761.995023:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760761.995132:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746688 00000100:00100000:0.0:1789760761.995142:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746688 00000100:00100000:0.0:1789760761.995205:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740382909668 time 740421255337 state 0x0 ref 1 00000100:00100000:0.0:1789760761.995220:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740421515753 time 740421255337 state 0x1 ref 1 00000100:00100000:0.0:1789760761.995252:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740421515753 time 740460541379 state 0x0 ref 1 00000100:00100000:0.0:1789760761.995265:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515746688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760761.995314:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23891968, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:0.0:1789760761.995324:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.995339:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760761.995343:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.995349:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.995360:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.048193:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23891968 -> 23896064 00010000:00010000:0.0:1789760762.048214:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.048226:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.048234:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.048240:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.048245:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.048253:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.048270:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.048278:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f1e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.048365:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515746688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53098us (53349us total) trans 4294973150 rc 0/0 00000100:00100000:0.0:1789760762.048375:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5862 00000100:00100000:0.0:1789760762.048384:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740421515753 time 740460541379 state 0x2 ref 0 00000001:00080000:2.0:1789760762.049050:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973150 is committed 00000100:00100000:3.0:1789760762.055153:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760762.056006:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746816 00000100:00100000:1.0:1789760762.056019:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746816 00000100:00100000:1.0:1789760762.056099:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740421515753 time 740460541379 state 0x0 ref 1 00000100:00100000:1.0:1789760762.056116:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740460801795 time 740460541379 state 0x1 ref 1 00000100:00080000:0.0:1789760762.056213:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760762.058862:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740460801795 time 740524138415 state 0x0 ref 1 00000100:00100000:1.0:1789760762.058892:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515746816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760762.058958:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23896064, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:1.0:1789760762.058967:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.058983:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760762.058987:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.058994:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059008:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760762.100102:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23896064 -> 23900160 00010000:00010000:2.0:1789760762.100120:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.100133:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.100143:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.100150:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.100155:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.100164:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.100187:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.100195:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f1f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.100279:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515746816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41395us (45130us total) trans 4294973151 rc 0/0 00000100:00100000:2.0:1789760762.100289:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5863 00000100:00100000:2.0:1789760762.100298:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740460801795 time 740524138415 state 0x2 ref 0 00000001:00080000:0.0:1789760762.100367:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973151 is committed 00000100:00100000:1.0:1789760762.103451:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.103588:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515746944 00000100:00100000:2.0:1789760762.103597:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515746944 00000100:00100000:2.0:1789760762.103646:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740460801795 time 740524138415 state 0x0 ref 1 00000100:00100000:2.0:1789760762.103660:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740524398831 time 740524138415 state 0x1 ref 1 00000100:00100000:2.0:1789760762.103739:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740524398831 time 740569028265 state 0x0 ref 1 00000100:00100000:2.0:1789760762.103759:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515746944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.103800:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23900160, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.103809:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.103823:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.103837:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.103844:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.103858:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.130837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973152 is committed 00002000:00010000:2.0:1789760762.130879:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23900160 -> 23904256 00010000:00010000:2.0:1789760762.130894:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.130907:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.130916:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.130923:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.130928:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.130937:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.130955:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.130968:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.131080:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515746944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27317us (27631us total) trans 4294973152 rc 0/0 00000100:00100000:2.0:1789760762.131089:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5864 00000100:00100000:2.0:1789760762.131099:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740524398831 time 740569028265 state 0x2 ref 0 00000100:00100000:1.0:1789760762.136637:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.136957:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747072 00000100:00100000:2.0:1789760762.136967:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747072 00000100:00100000:2.0:1789760762.137029:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740524398831 time 740569028265 state 0x0 ref 1 00000100:00100000:2.0:1789760762.137045:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740569288681 time 740569028265 state 0x1 ref 1 00000100:00100000:2.0:1789760762.137110:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740569288681 time 740602398726 state 0x0 ref 1 00000100:00100000:2.0:1789760762.137123:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515747072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.137178:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23904256, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.137187:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.137203:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.137207:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.137212:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.137230:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.174773:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760762.182864:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973153 is committed 00002000:00010000:2.0:1789760762.183105:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23904256 -> 23908352 00010000:00010000:2.0:1789760762.183121:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.183133:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.183142:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.183150:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.183154:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.183163:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.183181:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.183188:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f1fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.183281:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515747072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46155us (46647us total) trans 4294973153 rc 0/0 00000100:00100000:2.0:1789760762.183291:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5865 00000100:00100000:2.0:1789760762.183301:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740569288681 time 740602398726 state 0x2 ref 0 00000100:00100000:1.0:1789760762.187388:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.187467:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747200 00000100:00100000:2.0:1789760762.187478:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747200 00000100:00100000:2.0:1789760762.187540:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740569288681 time 740602398726 state 0x0 ref 1 00000100:00100000:2.0:1789760762.187556:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740602659142 time 740602398726 state 0x1 ref 1 00000100:00100000:2.0:1789760762.187589:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740602659142 time 740652878515 state 0x0 ref 1 00000100:00100000:2.0:1789760762.187601:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515747200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.187649:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23908352, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.187658:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.187673:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.187678:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.187684:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.187696:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.210567:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23908352 -> 23912448 00010000:00010000:3.0:1789760762.210658:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.210672:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973154 is committed 00010000:00010000:3.0:1789760762.210673:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.210687:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.210693:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760762.210698:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.210726:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.210744:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.210810:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f205 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.210957:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515747200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23350us (23570us total) trans 4294973154 rc 0/0 00000100:00100000:3.0:1789760762.210968:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5866 00000100:00100000:3.0:1789760762.210977:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740602659142 time 740652878515 state 0x2 ref 0 00000100:00100000:1.0:1789760762.214118:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760762.214294:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747328 00000100:00100000:3.0:1789760762.214301:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747328 00000100:00100000:3.0:1789760762.214545:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740602659142 time 740652878515 state 0x0 ref 1 00000100:00100000:3.0:1789760762.214609:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740653138931 time 740652878515 state 0x1 ref 1 00000100:00100000:3.0:1789760762.214688:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740653138931 time 740679977258 state 0x0 ref 1 00000100:00100000:3.0:1789760762.214703:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515747328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760762.214780:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23912448, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:3.0:1789760762.214789:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.214805:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760762.214809:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.214815:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.214840:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.241434:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23912448 -> 23916544 00010000:00010000:0.0:1789760762.241454:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.241456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973155 is committed 00010000:00010000:0.0:1789760762.241470:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.241480:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.241488:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.241493:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.241502:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.241522:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.241531:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f20c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.241852:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515747328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27148us (27739us total) trans 4294973155 rc 0/0 00000100:00100000:0.0:1789760762.241863:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5867 00000100:00100000:0.0:1789760762.241931:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740653138931 time 740679977258 state 0x2 ref 0 00000100:00100000:1.0:1789760762.245785:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760762.245947:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747456 00000100:00100000:0.0:1789760762.245958:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747456 00000100:00100000:0.0:1789760762.246009:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740653138931 time 740679977258 state 0x0 ref 1 00000100:00100000:0.0:1789760762.246025:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740680237674 time 740679977258 state 0x1 ref 1 00000100:00100000:0.0:1789760762.246046:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740680237674 time 740711335773 state 0x0 ref 1 00000100:00100000:0.0:1789760762.246064:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515747456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760762.246115:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23916544, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:0.0:1789760762.246124:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.246140:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760762.246145:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.246151:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.246171:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.266126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973156 is committed 00002000:00010000:0.0:1789760762.266624:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23916544 -> 23920640 00010000:00010000:0.0:1789760762.266637:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.266646:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.266652:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.266657:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.266660:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.266667:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.266682:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.266689:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f213 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.266850:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515747456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 20777us (21063us total) trans 4294973156 rc 0/0 00000100:00100000:0.0:1789760762.266861:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5868 00000100:00100000:0.0:1789760762.266870:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740680237674 time 740711335773 state 0x2 ref 0 00000100:00100000:1.0:1789760762.269969:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.270070:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747584 00000100:00100000:2.0:1789760762.270083:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747584 00000100:00100000:2.0:1789760762.270134:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740680237674 time 740711335773 state 0x0 ref 1 00000100:00100000:2.0:1789760762.270148:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740711596189 time 740711335773 state 0x1 ref 1 00000100:00100000:2.0:1789760762.270174:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740711596189 time 740735463332 state 0x0 ref 1 00000100:00100000:2.0:1789760762.270188:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515747584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.270232:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23920640, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.270239:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.270253:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.270257:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.270263:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.270277:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.309134:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23920640 -> 23924736 00000001:00080000:1.0:1789760762.309137:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973157 is committed 00010000:00010000:0.0:1789760762.309152:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.309165:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.309175:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.309182:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.309187:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.309196:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.309214:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.309225:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f21a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.309310:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515747584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39121us (39346us total) trans 4294973157 rc 0/0 00000100:00100000:0.0:1789760762.309320:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5869 00000100:00100000:0.0:1789760762.309330:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740711596189 time 740735463332 state 0x2 ref 0 00000100:00100000:1.0:1789760762.313853:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.313951:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747712 00000100:00100000:2.0:1789760762.313962:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747712 00000100:00100000:2.0:1789760762.314013:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740711596189 time 740735463332 state 0x0 ref 1 00000100:00100000:2.0:1789760762.314028:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740735723748 time 740735463332 state 0x1 ref 1 00000100:00100000:2.0:1789760762.314094:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740735723748 time 740779381262 state 0x0 ref 1 00000100:00100000:2.0:1789760762.314107:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515747712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.314161:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23924736, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.314170:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.314185:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.314190:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.314197:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.314211:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.358500:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973158 is committed 00002000:00010000:3.0:1789760762.358970:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23924736 -> 23928832 00010000:00010000:3.0:1789760762.358986:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.358998:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.359006:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.359013:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760762.359018:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.359027:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.359046:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.359054:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f221 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.359182:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515747712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45072us (45468us total) trans 4294973158 rc 0/0 00000100:00100000:3.0:1789760762.359193:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5870 00000100:00100000:3.0:1789760762.359203:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740735723748 time 740779381262 state 0x2 ref 0 00000100:00100000:1.0:1789760762.363835:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.364009:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747840 00000100:00100000:2.0:1789760762.364019:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747840 00000100:00100000:2.0:1789760762.364071:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740735723748 time 740779381262 state 0x0 ref 1 00000100:00100000:2.0:1789760762.364085:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740779641678 time 740779381262 state 0x1 ref 1 00000100:00100000:2.0:1789760762.364116:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740779641678 time 740829404991 state 0x0 ref 1 00000100:00100000:2.0:1789760762.364128:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515747840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.364186:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23928832, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.364195:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.364211:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.364215:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.364220:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.364233:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.367055:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93b10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544416000:0@lo:400:kworker.0 00000100:00100000:2.0:1789760762.367119:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760762.367171:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544416512:0@lo:400:kworker.0 00000100:00100000:2.0:1789760762.367198:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760762.367214:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416000 00000100:00100000:2.0:1789760762.367220:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760762.367220:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416000 00000100:00100000:2.0:1789760762.367225:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544416384:0@lo:400:kworker.0 00000100:00100000:2.0:1789760762.367248:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760762.367264:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:3.0:1789760762.367273:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544416000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760762.367327:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544416000:12345-0@lo:400:kworker.0 Request processed in 55us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.367335:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:3.0:1789760762.367374:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416512 00000100:00100000:3.0:1789760762.367378:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416512 00000100:00100000:3.0:1789760762.367409:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:3.0:1789760762.367415:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544416512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760762.367446:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544416512:12345-0@lo:400:kworker.0 Request processed in 31us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.367452:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:3.0:1789760762.367479:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416384 00000100:00100000:3.0:1789760762.367483:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416384 00000100:00100000:3.0:1789760762.367510:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:3.0:1789760762.367516:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544416384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760762.367545:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544416384:12345-0@lo:400:kworker.0 Request processed in 30us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.367552:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789760762.368025:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544416384:0@lo:400:kworker.0 00000100:00100000:2.0:1789760762.368036:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93b10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544416000:0@lo:400:kworker.0 00000100:00100000:2.0:1789760762.368044:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544416512:0@lo:400:kworker.0 00000100:00100000:0.0:1789760762.368341:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93b10b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544416128:0@lo:400:kworker.0 00000100:00100000:0.0:1789760762.368384:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760762.368435:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544416640:0@lo:400:kworker.0 00000100:00100000:0.0:1789760762.368457:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760762.368484:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416640 00000100:00100000:0.0:1789760762.368485:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760762.368489:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416640 00000100:00100000:0.0:1789760762.368491:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c023480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544416256:0@lo:400:kworker.0 00000100:00100000:0.0:1789760762.368511:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760762.368520:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:1.0:1789760762.368527:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544416640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760762.368566:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544416640:12345-0@lo:400:kworker.0 Request processed in 40us (108us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760762.368572:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:0.0:1789760762.369282:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544416640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760762.373568:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416128 00000100:00100000:3.0:1789760762.373581:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416128 00000100:00100000:3.0:1789760762.373633:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416256 00000100:00100000:3.0:1789760762.373636:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416256 00000100:00100000:3.0:1789760762.373660:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:3.0:1789760762.373668:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544416128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760762.373784:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:1.0:1789760762.373795:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544416256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760762.374052:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544416256:12345-0@lo:400:kworker.0 Request processed in 258us (5537us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.374052:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544416128:12345-0@lo:400:kworker.0 Request processed in 383us (5665us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.374060:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789760762.374197:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00080000:1.0:1789760762.374219:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760762.374223:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760762.374226:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760762.374281:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c023480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544416256:0@lo:400:kworker.0 00000100:00100000:1.0:1789760762.374296:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93b10b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544416128:0@lo:400:kworker.0 00000001:00080000:1.0:1789760762.407656:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973159 is committed 00002000:00010000:0.0:1789760762.407872:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23928832 -> 23932928 00010000:00010000:0.0:1789760762.407887:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.407900:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.407910:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.407918:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.407923:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.407933:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.407951:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.407958:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f228 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.408045:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515747840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43913us (44223us total) trans 4294973159 rc 0/0 00000100:00100000:0.0:1789760762.408054:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5871 00000100:00100000:0.0:1789760762.408065:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740779641678 time 740829404991 state 0x2 ref 0 00000100:00100000:0.0:1789760762.417470:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760762.417552:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515747968 00000100:00100000:3.0:1789760762.417561:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515747968 00000100:00100000:3.0:1789760762.417610:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740779641678 time 740829404991 state 0x0 ref 1 00000100:00100000:3.0:1789760762.417624:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740829665407 time 740829404991 state 0x1 ref 1 00000100:00100000:3.0:1789760762.417648:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740829665407 time 740882938116 state 0x0 ref 1 00000100:00100000:3.0:1789760762.417663:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515747968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760762.417934:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23932928, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:3.0:1789760762.417944:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.417960:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760762.417964:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.417970:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.417984:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760762.450930:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23932928 -> 23937024 00010000:00010000:2.0:1789760762.450950:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.450967:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.450976:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.450983:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.450988:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.450996:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.451013:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.451021:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f22f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.451111:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515747968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33443us (33644us total) trans 4294973160 rc 0/0 00000100:00100000:2.0:1789760762.451122:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5872 00000100:00100000:2.0:1789760762.451131:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740829665407 time 740882938116 state 0x2 ref 0 00000001:00080000:1.0:1789760762.451205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973160 is committed 00000100:00100000:1.0:1789760762.454986:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760762.455110:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748096 00000100:00100000:0.0:1789760762.455120:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748096 00000100:00100000:0.0:1789760762.455173:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740829665407 time 740882938116 state 0x0 ref 1 00000100:00100000:0.0:1789760762.455188:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740883198532 time 740882938116 state 0x1 ref 1 00000100:00100000:0.0:1789760762.455256:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740883198532 time 740920545241 state 0x0 ref 1 00000100:00100000:0.0:1789760762.455334:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515748096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760762.455385:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23937024, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:0.0:1789760762.455394:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.455410:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760762.455414:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.455421:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.455434:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.479651:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973161 is committed 00002000:00010000:3.0:1789760762.479897:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23937024 -> 23941120 00010000:00010000:3.0:1789760762.479913:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.479924:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.479932:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.479940:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760762.479945:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.479953:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.479972:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.479979:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f236 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.480107:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515748096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24772us (25126us total) trans 4294973161 rc 0/0 00000100:00100000:3.0:1789760762.480117:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5873 00000100:00100000:3.0:1789760762.480127:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740883198532 time 740920545241 state 0x2 ref 0 00000100:00100000:0.0:1789760762.484146:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760762.484239:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748224 00000100:00100000:1.0:1789760762.484249:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748224 00000100:00100000:1.0:1789760762.484304:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740883198532 time 740920545241 state 0x0 ref 1 00000100:00100000:1.0:1789760762.484319:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740920805657 time 740920545241 state 0x1 ref 1 00000100:00100000:1.0:1789760762.484350:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740920805657 time 740949638512 state 0x0 ref 1 00000100:00100000:1.0:1789760762.484362:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515748224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760762.484411:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23941120, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:1.0:1789760762.484421:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.484437:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760762.484441:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.484448:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.484461:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.509761:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23941120 -> 23945216 00010000:00010000:0.0:1789760762.509780:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.509793:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.509803:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.509810:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.509815:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.509833:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.509852:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.509860:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f23d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.509934:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515748224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25569us (25797us total) trans 4294973162 rc 0/0 00000100:00100000:0.0:1789760762.509943:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5874 00000100:00100000:0.0:1789760762.509952:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740920805657 time 740949638512 state 0x2 ref 0 00000001:00080000:3.0:1789760762.510260:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973162 is committed 00000100:00100000:0.0:1789760762.514638:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.514937:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748352 00000100:00100000:2.0:1789760762.514948:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748352 00000100:00100000:2.0:1789760762.515001:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740920805657 time 740949638512 state 0x0 ref 1 00000100:00100000:2.0:1789760762.515016:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740949898928 time 740949638512 state 0x1 ref 1 00000100:00100000:2.0:1789760762.515049:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740949898928 time 740980337860 state 0x0 ref 1 00000100:00100000:2.0:1789760762.515061:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515748352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.515111:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23945216, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.515121:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.515137:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760762.515141:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.515147:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.515160:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.549011:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973163 is committed 00002000:00010000:2.0:1789760762.549139:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23945216 -> 23949312 00010000:00010000:2.0:1789760762.549156:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.549168:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.549181:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.549188:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.549193:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.549202:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.549223:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.549231:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f244 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.549317:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515748352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34253us (34683us total) trans 4294973163 rc 0/0 00000100:00100000:2.0:1789760762.549327:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5875 00000100:00100000:2.0:1789760762.549336:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740949898928 time 740980337860 state 0x2 ref 0 00000100:00100000:1.0:1789760762.555481:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760762.555611:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748480 00000100:00100000:3.0:1789760762.555622:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748480 00000100:00100000:3.0:1789760762.555673:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740949898928 time 740980337860 state 0x0 ref 1 00000100:00100000:3.0:1789760762.555687:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740980598276 time 740980337860 state 0x1 ref 1 00000100:00100000:1.0:1789760762.555839:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740980598276 time 741021115125 state 0x0 ref 1 00000100:00100000:1.0:1789760762.555853:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515748480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760762.555907:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23949312, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:1.0:1789760762.555916:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.555932:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.555936:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.555942:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.555957:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.591128:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973164 is committed 00002000:00010000:1.0:1789760762.591257:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23949312 -> 23953408 00010000:00010000:1.0:1789760762.591273:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.591284:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.591292:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.591299:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760762.591304:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.591314:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.591333:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.591341:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f24b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.591437:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515748480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35583us (35961us total) trans 4294973164 rc 0/0 00000100:00100000:1.0:1789760762.591448:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5876 00000100:00100000:1.0:1789760762.591458:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740980598276 time 741021115125 state 0x2 ref 0 00000100:00100000:0.0:1789760762.602157:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.602247:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748608 00000100:00100000:2.0:1789760762.602258:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748608 00000100:00100000:2.0:1789760762.602310:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 740980598276 time 741021115125 state 0x0 ref 1 00000100:00100000:2.0:1789760762.602325:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741021375541 time 741021115125 state 0x1 ref 1 00000100:00100000:2.0:1789760762.602353:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741021375541 time 741067643242 state 0x0 ref 1 00000100:00100000:2.0:1789760762.602366:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515748608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.602411:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23953408, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.602419:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.602433:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.602437:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.602443:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.602457:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.681547:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973165 is committed 00002000:00010000:2.0:1789760762.681700:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23953408 -> 23957504 00010000:00010000:2.0:1789760762.681834:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.681847:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.681855:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.681863:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.681868:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.681877:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.681897:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.681906:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f252 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.682065:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515748608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 79696us (79912us total) trans 4294973165 rc 0/0 00000100:00100000:2.0:1789760762.682076:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5877 00000100:00100000:2.0:1789760762.682085:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741021375541 time 741067643242 state 0x2 ref 0 00000100:00100000:0.0:1789760762.700143:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.700244:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748736 00000100:00100000:2.0:1789760762.700254:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748736 00000100:00100000:2.0:1789760762.700306:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741021375541 time 741067643242 state 0x0 ref 1 00000100:00100000:2.0:1789760762.700322:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741067903658 time 741067643242 state 0x1 ref 1 00000100:00100000:2.0:1789760762.700386:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741067903658 time 741165674523 state 0x0 ref 1 00000100:00100000:2.0:1789760762.700398:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515748736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.700445:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23957504, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:2.0:1789760762.700454:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.700468:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.700473:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.700478:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.700491:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760762.738567:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23957504 -> 23961600 00010000:00010000:2.0:1789760762.738587:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.738600:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.738610:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.738617:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.738622:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.738631:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.738656:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.738664:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f259 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.738993:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515748736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38591us (38855us total) trans 4294973166 rc 0/0 00000100:00100000:3.0:1789760762.739003:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5878 00000100:00100000:3.0:1789760762.739012:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741067903658 time 741165674523 state 0x2 ref 0 00000001:00080000:0.0:1789760762.739360:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973166 is committed 00000100:00100000:1.0:1789760762.747061:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760762.747152:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748864 00000100:00100000:0.0:1789760762.747163:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748864 00000100:00100000:0.0:1789760762.747214:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741067903658 time 741165674523 state 0x0 ref 1 00000100:00100000:0.0:1789760762.747229:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741165934939 time 741165674523 state 0x1 ref 1 00000100:00100000:0.0:1789760762.747299:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741165934939 time 741212587349 state 0x0 ref 1 00000100:00100000:0.0:1789760762.747311:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515748864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760762.747358:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23961600, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:0.0:1789760762.747367:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.747381:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760762.747386:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.747392:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.747407:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.773214:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973167 is committed 00002000:00010000:0.0:1789760762.773271:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23961600 -> 23965696 00010000:00010000:0.0:1789760762.773285:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.773297:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.773306:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.773318:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.773322:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.773331:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.773348:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.773356:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f260 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.773480:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515748864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26166us (26425us total) trans 4294973167 rc 0/0 00000100:00100000:0.0:1789760762.773490:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5879 00000100:00100000:0.0:1789760762.773501:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741165934939 time 741212587349 state 0x2 ref 0 00000100:00100000:2.0:1789760762.777465:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760762.777542:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515748992 00000100:00100000:0.0:1789760762.777552:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515748992 00000100:00100000:0.0:1789760762.777607:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741165934939 time 741212587349 state 0x0 ref 1 00000100:00100000:0.0:1789760762.777622:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741212847765 time 741212587349 state 0x1 ref 1 00000100:00100000:0.0:1789760762.777651:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741212847765 time 741242940257 state 0x0 ref 1 00000100:00100000:0.0:1789760762.777667:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515748992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760762.777798:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23965696, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:0.0:1789760762.777807:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.777830:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760762.777835:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.777841:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.777855:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.802109:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760762.802148:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760762.802183:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760762.802212:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749376 00000100:00100000:1.0:1789760762.802221:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749376 00000100:00100000:1.0:1789760762.802274:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749248 00000100:00100000:1.0:1789760762.802277:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749248 00000100:00100000:1.0:1789760762.802299:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 476 00000100:00100000:1.0:1789760762.802309:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515749376:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760762.802377:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515749376:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 69us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760762.802385:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 476 00000100:00100000:1.0:1789760762.802401:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 477 00000100:00100000:1.0:1789760762.802408:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515749248:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760762.802431:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515749248:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 25us (249us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760762.802437:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 477 00000100:00080000:1.0:1789760762.802452:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760762.802654:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749120 00000100:00100000:2.0:1789760762.802658:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749120 00000100:00100000:2.0:1789760762.802695:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 400 00000100:00100000:2.0:1789760762.802702:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515749120:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760762.802767:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515749120:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 65us (667us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760762.802773:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 400 00000001:00080000:3.0:1789760762.834312:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973168 is committed 00002000:00010000:2.0:1789760762.834400:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23965696 -> 23969792 00010000:00010000:2.0:1789760762.834416:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.834428:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.834436:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.834442:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760762.834448:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.834456:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.834475:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.834482:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f267 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.834569:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515748992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56899us (57109us total) trans 4294973168 rc 0/0 00000100:00100000:2.0:1789760762.834579:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5880 00000100:00100000:2.0:1789760762.834589:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741212847765 time 741242940257 state 0x2 ref 0 00000100:00100000:3.0:1789760762.837562:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760762.838048:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749504 00000100:00100000:0.0:1789760762.838058:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749504 00000100:00100000:0.0:1789760762.838112:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741212847765 time 741242940257 state 0x0 ref 1 00000100:00100000:0.0:1789760762.838129:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741243200673 time 741242940257 state 0x1 ref 1 00000100:00100000:0.0:1789760762.838160:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741243200673 time 741303449400 state 0x0 ref 1 00000100:00100000:0.0:1789760762.838173:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515749504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760762.838218:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23969792, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb807 00010000:00010000:0.0:1789760762.838228:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.838248:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760762.838253:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.838260:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.838274:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.890866:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973169 is committed 00002000:00010000:3.0:1789760762.891911:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23969792 -> 23973888 00002000:00010000:3.0:1789760762.891918:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760762 -> 1789760763 00002000:00010000:3.0:1789760762.891921:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760762 -> 1789760763 00010000:00010000:3.0:1789760762.891935:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891947:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891956:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891964:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760762.891968:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891977:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.891996:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.892003:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f26e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.892155:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515749504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53979us (54597us total) trans 4294973169 rc 0/0 00000100:00100000:3.0:1789760762.892166:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5881 00000100:00100000:3.0:1789760762.892176:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741243200673 time 741303449400 state 0x2 ref 0 00000100:00100000:0.0:1789760762.903164:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760762.903312:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749632 00000100:00100000:1.0:1789760762.903322:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749632 00000100:00100000:1.0:1789760762.903379:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741243200673 time 741303449400 state 0x0 ref 1 00000100:00100000:1.0:1789760762.903394:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741303709816 time 741303449400 state 0x1 ref 1 00000100:00100000:1.0:1789760762.903459:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741303709816 time 741368747178 state 0x0 ref 1 00000100:00100000:1.0:1789760762.903471:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515749632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760762.903516:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23973888, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:1.0:1789760762.903525:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.903540:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.903544:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.903550:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.903569:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.933652:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973170 is committed 00002000:00010000:1.0:1789760762.933908:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23973888 -> 23977984 00010000:00010000:1.0:1789760762.933923:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.933935:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.933944:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.933952:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760762.933957:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.933966:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.933987:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.933995:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f275 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.934126:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515749632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30652us (30966us total) trans 4294973170 rc 0/0 00000100:00100000:1.0:1789760762.934137:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5882 00000100:00100000:1.0:1789760762.934146:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741303709816 time 741368747178 state 0x2 ref 0 00000100:00100000:1.0:1789760762.938369:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760762.938966:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749760 00000100:00100000:2.0:1789760762.938977:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749760 00000100:00100000:2.0:1789760762.939030:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741303709816 time 741368747178 state 0x0 ref 1 00000100:00100000:2.0:1789760762.939044:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741369007594 time 741368747178 state 0x1 ref 1 00000100:00100000:2.0:1789760762.939075:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741369007594 time 741404364029 state 0x0 ref 1 00000100:00100000:2.0:1789760762.939086:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515749760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760762.939131:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23977984, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:2.0:1789760762.939141:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.939157:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.939162:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.939167:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.939181:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760762.966610:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23977984 -> 23982080 00010000:00010000:0.0:1789760762.966631:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.966645:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.966655:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.966663:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760762.966668:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.966677:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.966697:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.966721:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f27c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.966816:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515749760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27727us (28451us total) trans 4294973171 rc 0/0 00000100:00100000:0.0:1789760762.966835:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5883 00000100:00100000:0.0:1789760762.966844:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741369007594 time 741404364029 state 0x2 ref 0 00000001:00080000:1.0:1789760762.967429:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973171 is committed 00000100:00100000:3.0:1789760762.970438:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760762.970527:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515749888 00000100:00100000:0.0:1789760762.970538:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515749888 00000100:00100000:0.0:1789760762.970588:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741369007594 time 741404364029 state 0x0 ref 1 00000100:00100000:0.0:1789760762.970604:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741404624445 time 741404364029 state 0x1 ref 1 00000100:00100000:0.0:1789760762.970662:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741404624445 time 741435951492 state 0x0 ref 1 00000100:00100000:0.0:1789760762.970674:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515749888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760762.970757:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23982080, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760762.970766:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.970782:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760762.970787:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.970792:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.970806:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.004148:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23982080 -> 23986176 00010000:00010000:0.0:1789760763.004168:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.004181:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.004190:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.004190:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973172 is committed 00010000:00010000:0.0:1789760763.004197:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.004202:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.004211:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.004231:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.004246:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f283 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.004399:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515749888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33714us (33958us total) trans 4294973172 rc 0/0 00000100:00100000:0.0:1789760763.004410:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5884 00000100:00100000:0.0:1789760763.004419:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741404624445 time 741435951492 state 0x2 ref 0 00000100:00100000:3.0:1789760763.008661:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.008926:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750016 00000100:00100000:0.0:1789760763.008935:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750016 00000100:00100000:0.0:1789760763.008989:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741404624445 time 741435951492 state 0x0 ref 1 00000100:00100000:0.0:1789760763.009004:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741436211908 time 741435951492 state 0x1 ref 1 00000100:00100000:0.0:1789760763.009023:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741436211908 time 741474312912 state 0x0 ref 1 00000100:00100000:0.0:1789760763.009035:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515750016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.009082:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23986176, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.009091:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.009106:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760763.009110:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.009117:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.009135:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.053607:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973173 is committed 00002000:00010000:0.0:1789760763.053727:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23986176 -> 23990272 00010000:00010000:0.0:1789760763.055600:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.055617:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.055626:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.055633:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.055637:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.055646:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.055667:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.055674:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f28a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.056189:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515750016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47150us (47531us total) trans 4294973173 rc 0/0 00000100:00100000:0.0:1789760763.056200:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5885 00000100:00100000:0.0:1789760763.056210:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741436211908 time 741474312912 state 0x2 ref 0 00000100:00100000:1.0:1789760763.065629:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.065743:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750144 00000100:00100000:0.0:1789760763.065753:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750144 00000100:00100000:0.0:1789760763.065830:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741436211908 time 741474312912 state 0x0 ref 1 00000100:00100000:0.0:1789760763.065846:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741474573328 time 741474312912 state 0x1 ref 1 00000100:00100000:0.0:1789760763.065919:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741474573328 time 741531207712 state 0x0 ref 1 00000100:00100000:0.0:1789760763.065941:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515750144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.065988:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23990272, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.065998:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.066014:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760763.066018:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.066024:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.066039:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.115063:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973174 is committed 00002000:00010000:1.0:1789760763.115128:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23990272 -> 23994368 00010000:00010000:1.0:1789760763.115143:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.115156:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.115165:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.115184:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.115189:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.115198:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.115218:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.115226:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f291 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.115320:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515750144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49377us (49695us total) trans 4294973174 rc 0/0 00000100:00100000:1.0:1789760763.115330:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5886 00000100:00100000:1.0:1789760763.115340:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741474573328 time 741531207712 state 0x2 ref 0 00000100:00100000:1.0:1789760763.132092:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760763.132308:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750272 00000100:00100000:3.0:1789760763.132319:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750272 00000100:00100000:3.0:1789760763.132371:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741474573328 time 741531207712 state 0x0 ref 1 00000100:00100000:3.0:1789760763.132387:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741531468128 time 741531207712 state 0x1 ref 1 00000100:00100000:3.0:1789760763.132418:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741531468128 time 741597706557 state 0x0 ref 1 00000100:00100000:3.0:1789760763.132430:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515750272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760763.132478:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23994368, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:3.0:1789760763.132487:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.132503:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760763.132507:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.132513:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.132532:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.191088:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23994368 -> 23998464 00010000:00010000:3.0:1789760763.191108:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.191120:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.191128:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.191135:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760763.191140:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.191148:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.191166:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.191173:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f298 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.191830:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515750272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59387us (59734us total) trans 4294973175 rc 0/0 00000100:00100000:3.0:1789760763.191841:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5887 00000100:00100000:3.0:1789760763.191852:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741531468128 time 741597706557 state 0x2 ref 0 00000001:00080000:1.0:1789760763.192039:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973175 is committed 00000100:00100000:0.0:1789760763.197306:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760763.197372:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750400 00000100:00100000:1.0:1789760763.197383:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750400 00000100:00100000:1.0:1789760763.197438:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741531468128 time 741597706557 state 0x0 ref 1 00000100:00100000:1.0:1789760763.197454:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741597966973 time 741597706557 state 0x1 ref 1 00000100:00100000:1.0:1789760763.197491:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741597966973 time 741662780000 state 0x0 ref 1 00000100:00100000:1.0:1789760763.197504:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515750400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760763.197554:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 23998464, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:1.0:1789760763.197562:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.197583:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760763.197588:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.197594:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.197607:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.197898:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760763.244974:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973176 is committed 00002000:00010000:1.0:1789760763.245381:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 23998464 -> 24002560 00010000:00010000:1.0:1789760763.245400:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245414:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245423:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245431:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.245435:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245444:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245464:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.245473:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f29f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.245568:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515750400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48061us (48266us total) trans 4294973176 rc 0/0 00000100:00100000:1.0:1789760763.245579:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5888 00000100:00100000:1.0:1789760763.245588:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741597966973 time 741662780000 state 0x2 ref 0 00000100:00100000:2.0:1789760763.249733:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.250087:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750528 00000100:00100000:0.0:1789760763.250097:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750528 00000100:00100000:0.0:1789760763.250150:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741597966973 time 741662780000 state 0x0 ref 1 00000100:00100000:0.0:1789760763.251477:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741663040416 time 741662780000 state 0x1 ref 1 00000100:00100000:0.0:1789760763.251535:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741663040416 time 741716825308 state 0x0 ref 1 00000100:00100000:0.0:1789760763.251550:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515750528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.251593:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24002560, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.251600:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.251612:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760763.251616:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.251620:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.251660:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.286753:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24002560 -> 24006656 00010000:00010000:2.0:1789760763.286774:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.286787:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.286797:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.286805:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760763.286811:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.286838:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.286859:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.286867:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f2a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.286947:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515750528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35398us (37247us total) trans 4294973177 rc 0/0 00000100:00100000:2.0:1789760763.286958:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5889 00000100:00100000:2.0:1789760763.286967:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741663040416 time 741716825308 state 0x2 ref 0 00000001:00080000:2.0:1789760763.287007:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973177 is committed 00000100:00100000:3.0:1789760763.292003:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.292161:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750656 00000100:00100000:0.0:1789760763.292172:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750656 00000100:00100000:0.0:1789760763.292224:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741663040416 time 741716825308 state 0x0 ref 1 00000100:00100000:0.0:1789760763.292240:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741717085724 time 741716825308 state 0x1 ref 1 00000100:00100000:0.0:1789760763.292270:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741717085724 time 741757559677 state 0x0 ref 1 00000100:00100000:0.0:1789760763.292282:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515750656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.292333:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24006656, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.292342:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.292359:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760763.292363:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.292370:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.292383:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.332081:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973178 is committed 00002000:00010000:0.0:1789760763.332168:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24006656 -> 24010752 00010000:00010000:0.0:1789760763.332183:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.332196:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.332205:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.332218:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.332223:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.332232:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.332251:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.332258:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f2ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.332484:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515750656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40197us (40485us total) trans 4294973178 rc 0/0 00000100:00100000:0.0:1789760763.332496:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5890 00000100:00100000:0.0:1789760763.332506:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741717085724 time 741757559677 state 0x2 ref 0 00000100:00100000:0.0:1789760763.341232:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760763.341330:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750784 00000100:00100000:2.0:1789760763.341340:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750784 00000100:00100000:2.0:1789760763.341391:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741717085724 time 741757559677 state 0x0 ref 1 00000100:00100000:2.0:1789760763.341407:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741757820093 time 741757559677 state 0x1 ref 1 00000100:00100000:2.0:1789760763.341492:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741757820093 time 741806779790 state 0x0 ref 1 00000100:00100000:2.0:1789760763.341505:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515750784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760763.341553:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24010752, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:2.0:1789760763.341562:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.341578:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760763.341583:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.341589:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.341603:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.367017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973179 is committed 00002000:00010000:2.0:1789760763.367308:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24010752 -> 24014848 00010000:00010000:2.0:1789760763.367324:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.367336:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.367344:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.367351:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760763.367355:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.367364:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.367383:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.367390:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f2b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.367475:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515750784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25967us (26251us total) trans 4294973179 rc 0/0 00000100:00100000:2.0:1789760763.367484:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5891 00000100:00100000:2.0:1789760763.367493:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741757820093 time 741806779790 state 0x2 ref 0 00000100:00100000:2.0:1789760763.373172:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760763.373261:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515750912 00000100:00100000:1.0:1789760763.373273:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515750912 00000100:00100000:1.0:1789760763.373325:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741757820093 time 741806779790 state 0x0 ref 1 00000100:00100000:1.0:1789760763.373341:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741807040206 time 741806779790 state 0x1 ref 1 00000100:00100000:1.0:1789760763.373371:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741807040206 time 741838660293 state 0x0 ref 1 00000100:00100000:1.0:1789760763.373383:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515750912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760763.373431:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24014848, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:1.0:1789760763.373440:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.373455:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760763.373471:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.373487:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.373507:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.401193:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973180 is committed 00002000:00010000:1.0:1789760763.401317:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24014848 -> 24018944 00010000:00010000:1.0:1789760763.401332:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.401345:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.401355:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.401363:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.401368:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.401377:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.401397:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.401405:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f2bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.401568:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515750912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28181us (28400us total) trans 4294973180 rc 0/0 00000100:00100000:1.0:1789760763.401579:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5892 00000100:00100000:1.0:1789760763.401588:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741807040206 time 741838660293 state 0x2 ref 0 00000100:00100000:3.0:1789760763.404784:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760763.404833:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751040 00000100:00100000:3.0:1789760763.404841:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751040 00000100:00100000:3.0:1789760763.404890:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741807040206 time 741838660293 state 0x0 ref 1 00000100:00100000:3.0:1789760763.404904:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741838920709 time 741838660293 state 0x1 ref 1 00000100:00100000:3.0:1789760763.404925:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741838920709 time 741870214524 state 0x0 ref 1 00000100:00100000:3.0:1789760763.404937:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515751040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760763.404985:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24018944, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:3.0:1789760763.404994:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.405009:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760763.405014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.405020:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.405040:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760763.432904:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24018944 -> 24023040 00010000:00010000:0.0:1789760763.432924:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.432944:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.432953:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.432960:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.432965:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.432975:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.432994:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.433002:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f2c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.433096:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515751040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28156us (28317us total) trans 4294973181 rc 0/0 00000100:00100000:0.0:1789760763.433106:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5893 00000100:00100000:0.0:1789760763.433116:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741838920709 time 741870214524 state 0x2 ref 0 00000001:00080000:1.0:1789760763.437073:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973181 is committed 00000100:00100000:0.0:1789760763.437853:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.437895:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751168 00000100:00100000:0.0:1789760763.437904:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751168 00000100:00100000:0.0:1789760763.437952:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741838920709 time 741870214524 state 0x0 ref 1 00000100:00100000:0.0:1789760763.437967:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741870474940 time 741870214524 state 0x1 ref 1 00000100:00100000:0.0:1789760763.437992:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741870474940 time 741903281465 state 0x0 ref 1 00000100:00100000:0.0:1789760763.438005:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.438052:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24023040, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.438061:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.438076:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760763.438081:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.438086:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.438099:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.467229:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24023040 -> 24027136 00010000:00010000:1.0:1789760763.467249:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.467263:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.467272:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.467280:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.467285:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.467294:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.467315:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.467326:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.467429:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515751168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29421us (29576us total) trans 4294973182 rc 0/0 00000100:00100000:1.0:1789760763.467440:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5894 00000100:00100000:1.0:1789760763.467450:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741870474940 time 741903281465 state 0x2 ref 0 00000001:00080000:3.0:1789760763.467606:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973182 is committed 00000100:00100000:2.0:1789760763.470938:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.471029:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751296 00000100:00100000:0.0:1789760763.471039:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751296 00000100:00100000:0.0:1789760763.471089:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741870474940 time 741903281465 state 0x0 ref 1 00000100:00100000:0.0:1789760763.471104:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741903541881 time 741903281465 state 0x1 ref 1 00000100:00100000:0.0:1789760763.471135:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741903541881 time 741936424413 state 0x0 ref 1 00000100:00100000:0.0:1789760763.471148:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515751296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.471194:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24027136, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.471202:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.471218:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760763.471223:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.471229:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.471243:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.492105:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973183 is committed 00002000:00010000:0.0:1789760763.492214:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24027136 -> 24031232 00010000:00010000:0.0:1789760763.492228:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.492241:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.492250:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.492257:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.492262:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.492271:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.492300:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.492309:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f2d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.492398:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515751296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21247us (21464us total) trans 4294973183 rc 0/0 00000100:00100000:0.0:1789760763.492408:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5895 00000100:00100000:0.0:1789760763.492418:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741903541881 time 741936424413 state 0x2 ref 0 00000100:00100000:0.0:1789760763.496441:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760763.496538:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751424 00000100:00100000:2.0:1789760763.496548:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751424 00000100:00100000:2.0:1789760763.496599:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741903541881 time 741936424413 state 0x0 ref 1 00000100:00100000:2.0:1789760763.496613:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741936684829 time 741936424413 state 0x1 ref 1 00000100:00100000:2.0:1789760763.496642:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741936684829 time 741961930845 state 0x0 ref 1 00000100:00100000:2.0:1789760763.496654:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caa850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760763.496702:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24031232, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:2.0:1789760763.496752:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.496775:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760763.496779:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.496785:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.496798:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.519482:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973184 is committed 00002000:00010000:2.0:1789760763.519546:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24031232 -> 24035328 00010000:00010000:2.0:1789760763.519560:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.519572:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.519581:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.519588:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760763.519592:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.519601:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.519625:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.519632:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f2d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.519757:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caa850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23053us (23274us total) trans 4294973184 rc 0/0 00000100:00100000:2.0:1789760763.519767:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5896 00000100:00100000:2.0:1789760763.519777:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741936684829 time 741961930845 state 0x2 ref 0 00000100:00100000:0.0:1789760763.527919:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760763.528010:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751552 00000100:00100000:2.0:1789760763.528021:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751552 00000100:00100000:2.0:1789760763.528075:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741936684829 time 741961930845 state 0x0 ref 1 00000100:00100000:2.0:1789760763.528090:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741962191261 time 741961930845 state 0x1 ref 1 00000100:00100000:2.0:1789760763.528112:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741962191261 time 741993401508 state 0x0 ref 1 00000100:00100000:2.0:1789760763.528124:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515751552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760763.528172:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24035328, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:2.0:1789760763.528180:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.528195:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760763.528200:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.528207:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.528220:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.556029:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24035328 -> 24039424 00010000:00010000:2.0:1789760763.556048:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.556062:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.556071:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.556079:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760763.556083:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.556094:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.556117:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.556130:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f2de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.556223:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515751552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28097us (28309us total) trans 4294973185 rc 0/0 00000100:00100000:2.0:1789760763.556234:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5897 00000100:00100000:2.0:1789760763.556244:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741962191261 time 741993401508 state 0x2 ref 0 00000001:00080000:3.0:1789760763.559903:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973185 is committed 00000100:00100000:2.0:1789760763.560528:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760763.560644:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751680 00000100:00100000:1.0:1789760763.560655:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751680 00000100:00100000:1.0:1789760763.560752:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741962191261 time 741993401508 state 0x0 ref 1 00000100:00100000:1.0:1789760763.560769:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741993661924 time 741993401508 state 0x1 ref 1 00000100:00100000:1.0:1789760763.560792:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741993661924 time 742026081208 state 0x0 ref 1 00000100:00100000:1.0:1789760763.560805:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760763.560865:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24039424, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:1.0:1789760763.560874:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.560892:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.560897:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.560902:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.560921:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.589044:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24039424 -> 24043520 00010000:00010000:1.0:1789760763.589064:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.589077:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.589086:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.589093:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.589098:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.589107:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.589127:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.589136:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f2e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.589224:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515751680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28415us (28695us total) trans 4294973186 rc 0/0 00000100:00100000:1.0:1789760763.589234:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5898 00000100:00100000:1.0:1789760763.589244:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741993661924 time 742026081208 state 0x2 ref 0 00000001:00080000:1.0:1789760763.589282:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973186 is committed 00000100:00100000:1.0:1789760763.592779:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.592950:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751808 00000100:00100000:0.0:1789760763.592960:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751808 00000100:00100000:0.0:1789760763.593011:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 741993661924 time 742026081208 state 0x0 ref 1 00000100:00100000:0.0:1789760763.593027:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742026341624 time 742026081208 state 0x1 ref 1 00000100:00100000:0.0:1789760763.593059:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742026341624 time 742058347890 state 0x0 ref 1 00000100:00100000:0.0:1789760763.593072:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.593115:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24043520, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.593121:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.593134:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.593138:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.593142:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.593152:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.614653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973187 is committed 00002000:00010000:1.0:1789760763.614995:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24043520 -> 24047616 00010000:00010000:1.0:1789760763.615010:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615020:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615027:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615031:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.615036:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615043:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615061:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615068:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f2ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.615215:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22140us (22441us total) trans 4294973187 rc 0/0 00000100:00100000:1.0:1789760763.615226:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5899 00000100:00100000:1.0:1789760763.615235:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742026341624 time 742058347890 state 0x2 ref 0 00000100:00100000:2.0:1789760763.618327:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.618494:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515751936 00000100:00100000:0.0:1789760763.618512:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515751936 00000100:00100000:0.0:1789760763.618593:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742026341624 time 742058347890 state 0x0 ref 1 00000100:00100000:0.0:1789760763.618613:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742058608306 time 742058347890 state 0x1 ref 1 00000100:00100000:0.0:1789760763.618651:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742058608306 time 742083940250 state 0x0 ref 1 00000100:00100000:0.0:1789760763.618664:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.618772:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24047616, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.618782:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.618799:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.618805:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.618813:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.618842:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.647653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973188 is committed 00002000:00010000:0.0:1789760763.647808:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24047616 -> 24051712 00010000:00010000:0.0:1789760763.647834:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.648001:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.648013:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.648020:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.648025:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.648034:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.648058:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.648066:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f2f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.648169:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515751936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29503us (29852us total) trans 4294973188 rc 0/0 00000100:00100000:0.0:1789760763.648179:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5900 00000100:00100000:0.0:1789760763.648189:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742058608306 time 742083940250 state 0x2 ref 0 00000100:00100000:2.0:1789760763.651277:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.652003:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752064 00000100:00100000:0.0:1789760763.652015:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752064 00000100:00100000:1.0:1789760763.652113:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742058608306 time 742083940250 state 0x0 ref 1 00000100:00100000:1.0:1789760763.652131:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742084200666 time 742083940250 state 0x1 ref 1 00000100:00100000:1.0:1789760763.652171:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742084200666 time 742117459780 state 0x0 ref 1 00000100:00100000:1.0:1789760763.652186:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760763.652241:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24051712, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:1.0:1789760763.652252:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.652273:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.652278:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.652287:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.652310:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.697226:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24051712 -> 24055808 00010000:00010000:1.0:1789760763.697247:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.697261:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.697271:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.697278:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.697283:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.697292:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.697311:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.697320:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f2fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.697414:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515752064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45227us (46141us total) trans 4294973189 rc 0/0 00000100:00100000:1.0:1789760763.697424:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5901 00000100:00100000:1.0:1789760763.697434:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742084200666 time 742117459780 state 0x2 ref 0 00000001:00080000:1.0:1789760763.697471:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973189 is committed 00000100:00100000:1.0:1789760763.704927:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760763.705936:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752192 00000100:00100000:2.0:1789760763.705949:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752192 00000100:00100000:3.0:1789760763.706106:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742084200666 time 742117459780 state 0x0 ref 1 00000100:00100000:3.0:1789760763.706123:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742117720196 time 742117459780 state 0x1 ref 1 00000100:00100000:3.0:1789760763.706157:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742117720196 time 742171445661 state 0x0 ref 1 00000100:00100000:3.0:1789760763.706171:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515752192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760763.706220:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24055808, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:3.0:1789760763.706229:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.706246:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.706250:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.706256:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.706271:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.738387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973190 is committed 00002000:00010000:0.0:1789760763.738491:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24055808 -> 24059904 00010000:00010000:0.0:1789760763.738506:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.738519:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.738528:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.738536:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.738540:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.738550:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.738570:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.738583:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f301 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.738680:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515752192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32507us (33759us total) trans 4294973190 rc 0/0 00000100:00100000:0.0:1789760763.738691:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5902 00000100:00100000:0.0:1789760763.738702:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742117720196 time 742171445661 state 0x2 ref 0 00000100:00100000:1.0:1789760763.742831:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760763.742992:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752320 00000100:00100000:2.0:1789760763.743002:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752320 00000100:00100000:2.0:1789760763.743057:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742117720196 time 742171445661 state 0x0 ref 1 00000100:00100000:2.0:1789760763.743072:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742171706077 time 742171445661 state 0x1 ref 1 00000100:00100000:2.0:1789760763.743103:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742171706077 time 742208391675 state 0x0 ref 1 00000100:00100000:2.0:1789760763.743116:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760763.743165:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24059904, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:2.0:1789760763.743175:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.743192:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760763.743197:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.743203:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.743218:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.772863:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973191 is committed 00002000:00010000:0.0:1789760763.772970:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24059904 -> 24064000 00010000:00010000:0.0:1789760763.772986:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.773000:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.773009:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.773016:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.773020:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.773029:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.773052:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.773059:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f308 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.773185:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30067us (30368us total) trans 4294973191 rc 0/0 00000100:00100000:0.0:1789760763.773196:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5903 00000100:00100000:0.0:1789760763.773205:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742171706077 time 742208391675 state 0x2 ref 0 00000100:00100000:1.0:1789760763.776326:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760763.776390:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752448 00000100:00100000:2.0:1789760763.776401:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752448 00000100:00100000:2.0:1789760763.776835:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742171706077 time 742208391675 state 0x0 ref 1 00000100:00100000:2.0:1789760763.776853:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742208652091 time 742208391675 state 0x1 ref 1 00000100:00100000:2.0:1789760763.776886:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742208652091 time 742242175268 state 0x0 ref 1 00000100:00100000:2.0:1789760763.776899:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515752448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760763.776946:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24064000, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:2.0:1789760763.776954:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.776969:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760763.776973:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.776978:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.776995:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.807606:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24064000 -> 24068096 00010000:00010000:1.0:1789760763.807628:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.807652:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.807660:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.807667:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.807672:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.807682:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.807703:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.807749:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f30f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.807848:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515752448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30947us (31525us total) trans 4294973192 rc 0/0 00000100:00100000:1.0:1789760763.807858:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5904 00000100:00100000:1.0:1789760763.807868:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742208652091 time 742242175268 state 0x2 ref 0 00000001:00080000:3.0:1789760763.810937:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973192 is committed 00000100:00100000:1.0:1789760763.814260:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.814434:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752576 00000100:00100000:0.0:1789760763.814444:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752576 00000100:00100000:0.0:1789760763.814499:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742208652091 time 742242175268 state 0x0 ref 1 00000100:00100000:0.0:1789760763.814514:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742242435684 time 742242175268 state 0x1 ref 1 00000100:00100000:0.0:1789760763.814609:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742242435684 time 742279890271 state 0x0 ref 1 00000100:00100000:0.0:1789760763.814622:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.814677:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24068096, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb807 00010000:00010000:0.0:1789760763.814687:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.814703:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.814726:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.814733:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.814746:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.859132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973193 is committed 00002000:00010000:0.0:1789760763.859293:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24068096 -> 24072192 00002000:00010000:0.0:1789760763.859298:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760763 -> 1789760764 00002000:00010000:0.0:1789760763.859300:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760763 -> 1789760764 00010000:00010000:0.0:1789760763.859315:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.859326:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.859335:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.859343:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760763.859348:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.859357:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.859377:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.859396:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f316 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.859489:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44864us (45233us total) trans 4294973193 rc 0/0 00000100:00100000:0.0:1789760763.859499:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5905 00000100:00100000:0.0:1789760763.859509:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742242435684 time 742279890271 state 0x2 ref 0 00000100:00100000:1.0:1789760763.869201:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760763.869841:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752704 00000100:00100000:0.0:1789760763.869851:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752704 00000100:00100000:0.0:1789760763.869903:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742242435684 time 742279890271 state 0x0 ref 1 00000100:00100000:0.0:1789760763.869918:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742280150687 time 742279890271 state 0x1 ref 1 00000100:00100000:0.0:1789760763.869950:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742280150687 time 742335239382 state 0x0 ref 1 00000100:00100000:0.0:1789760763.869961:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515752704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760763.870006:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24072192, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:0.0:1789760763.870015:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.870030:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.870034:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.870041:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.870060:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.912536:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24072192 -> 24076288 00010000:00010000:2.0:1789760763.912558:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.912570:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.912579:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.912586:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760763.912591:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.912600:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.912620:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.912628:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f31d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.912722:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973194 is committed 00000100:00100000:2.0:1789760763.912757:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515752704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42792us (43559us total) trans 4294973194 rc 0/0 00000100:00100000:2.0:1789760763.912767:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5906 00000100:00100000:2.0:1789760763.912776:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742280150687 time 742335239382 state 0x2 ref 0 00000100:00100000:2.0:1789760763.917903:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760763.917992:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752832 00000100:00100000:1.0:1789760763.918003:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752832 00000100:00100000:1.0:1789760763.918055:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742280150687 time 742335239382 state 0x0 ref 1 00000100:00100000:1.0:1789760763.918070:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742335499798 time 742335239382 state 0x1 ref 1 00000100:00100000:1.0:1789760763.918142:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742335499798 time 742383430148 state 0x0 ref 1 00000100:00100000:1.0:1789760763.918154:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760763.918200:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24076288, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:1.0:1789760763.918209:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.918231:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.918235:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.918242:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.918255:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.960154:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973195 is committed 00002000:00010000:2.0:1789760763.960247:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24076288 -> 24080384 00010000:00010000:2.0:1789760763.960264:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.960275:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.960285:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.960292:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760763.960296:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.960306:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.960324:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.960337:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f324 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.960459:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515752832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42302us (42559us total) trans 4294973195 rc 0/0 00000100:00100000:2.0:1789760763.960470:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5907 00000100:00100000:2.0:1789760763.960479:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742335499798 time 742383430148 state 0x2 ref 0 00000100:00100000:0.0:1789760763.964402:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760763.964502:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515752960 00000100:00100000:1.0:1789760763.964512:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515752960 00000100:00100000:1.0:1789760763.964565:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742335499798 time 742383430148 state 0x0 ref 1 00000100:00100000:1.0:1789760763.964579:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742383690564 time 742383430148 state 0x1 ref 1 00000100:00100000:1.0:1789760763.964609:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742383690564 time 742429898634 state 0x0 ref 1 00000100:00100000:1.0:1789760763.964621:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515752960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760763.964667:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24080384, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:1.0:1789760763.964675:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.964691:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.964695:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.964701:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.964757:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.993920:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24080384 -> 24084480 00010000:00010000:1.0:1789760763.993949:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.993961:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.993971:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.993978:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760763.993983:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.993992:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.994011:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.994020:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f32b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.994122:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515752960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29498us (29726us total) trans 4294973196 rc 0/0 00000100:00100000:1.0:1789760763.994133:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5908 00000100:00100000:1.0:1789760763.994143:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742383690564 time 742429898634 state 0x2 ref 0 00000001:00080000:0.0:1789760763.994970:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973196 is committed 00000100:00100000:0.0:1789760764.000339:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760764.000428:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753088 00000100:00100000:2.0:1789760764.000437:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753088 00000100:00100000:2.0:1789760764.000487:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742383690564 time 742429898634 state 0x0 ref 1 00000100:00100000:2.0:1789760764.000503:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742430159050 time 742429898634 state 0x1 ref 1 00000100:00100000:2.0:1789760764.002159:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742430159050 time 742467437493 state 0x0 ref 1 00000100:00100000:2.0:1789760764.002198:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760764.002262:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24084480, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:2.0:1789760764.002272:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.002288:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760764.002292:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.002299:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.002318:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760764.002959:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760764.042013:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973197 is committed 00002000:00010000:0.0:1789760764.042123:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24084480 -> 24088576 00010000:00010000:0.0:1789760764.042139:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.042152:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.042160:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.042168:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.042182:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.042192:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.042213:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.042222:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f332 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.042351:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40170us (42015us total) trans 4294973197 rc 0/0 00000100:00100000:0.0:1789760764.042362:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5909 00000100:00100000:0.0:1789760764.042372:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742430159050 time 742467437493 state 0x2 ref 0 00000100:00100000:0.0:1789760764.048961:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760764.049053:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753216 00000100:00100000:2.0:1789760764.049063:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753216 00000100:00100000:2.0:1789760764.049172:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742430159050 time 742467437493 state 0x0 ref 1 00000100:00100000:2.0:1789760764.049188:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742467697909 time 742467437493 state 0x1 ref 1 00000100:00100000:2.0:1789760764.049221:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742467697909 time 742514510607 state 0x0 ref 1 00000100:00100000:2.0:1789760764.049235:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515753216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.051206:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24088576, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:3.0:1789760764.051220:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.051236:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760764.051240:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.051246:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.051312:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.100111:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973198 is committed 00002000:00010000:0.0:1789760764.100252:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24088576 -> 24092672 00010000:00010000:0.0:1789760764.100268:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.100282:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.100291:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.100297:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.100302:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.100311:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.100334:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.100342:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f339 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.100436:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515753216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51197us (51478us total) trans 4294973198 rc 0/0 00000100:00100000:0.0:1789760764.100448:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5910 00000100:00100000:0.0:1789760764.100459:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742467697909 time 742514510607 state 0x2 ref 0 00000100:00100000:1.0:1789760764.117204:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760764.117644:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753344 00000100:00100000:0.0:1789760764.117654:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753344 00000100:00100000:0.0:1789760764.117848:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742467697909 time 742514510607 state 0x0 ref 1 00000100:00100000:0.0:1789760764.117868:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742514771023 time 742514510607 state 0x1 ref 1 00000100:00100000:1.0:1789760764.117964:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742514771023 time 742583250875 state 0x0 ref 1 00000100:00100000:1.0:1789760764.117979:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515753344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760764.118030:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24092672, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:1.0:1789760764.118039:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.118057:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760764.118062:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.118068:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.118082:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.157538:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973199 is committed 00002000:00010000:1.0:1789760764.157556:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24092672 -> 24096768 00010000:00010000:1.0:1789760764.157572:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.157588:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.157598:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.157605:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760764.157610:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.157619:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.157639:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.157647:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f340 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.157811:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515753344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39829us (40610us total) trans 4294973199 rc 0/0 00000100:00100000:1.0:1789760764.157832:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5911 00000100:00100000:1.0:1789760764.157842:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742514771023 time 742583250875 state 0x2 ref 0 00000100:00100000:1.0:1789760764.161989:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760764.163697:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753472 00000100:00100000:0.0:1789760764.163763:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753472 00000100:00100000:0.0:1789760764.163831:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742514771023 time 742583250875 state 0x0 ref 1 00000100:00100000:0.0:1789760764.163848:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742583511291 time 742583250875 state 0x1 ref 1 00000100:00100000:0.0:1789760764.163919:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742583511291 time 742629207423 state 0x0 ref 1 00000100:00100000:0.0:1789760764.163932:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760764.163983:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24096768, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:0.0:1789760764.163992:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.164009:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760764.164014:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.164021:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.164036:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760764.203632:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24096768 -> 24100864 00010000:00010000:2.0:1789760764.203654:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.203666:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.203675:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.203681:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760764.203687:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.203695:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.203747:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.203755:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f347 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.203868:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515753472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39933us (41883us total) trans 4294973200 rc 0/0 00000100:00100000:2.0:1789760764.203879:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5912 00000100:00100000:2.0:1789760764.203889:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742583511291 time 742629207423 state 0x2 ref 0 00000001:00080000:1.0:1789760764.204077:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973200 is committed 00000100:00100000:0.0:1789760764.211501:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760764.211587:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753600 00000100:00100000:1.0:1789760764.211597:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753600 00000100:00100000:1.0:1789760764.211651:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742583511291 time 742629207423 state 0x0 ref 1 00000100:00100000:1.0:1789760764.211667:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742629467839 time 742629207423 state 0x1 ref 1 00000100:00100000:1.0:1789760764.211695:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742629467839 time 742676984418 state 0x0 ref 1 00000100:00100000:1.0:1789760764.211744:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760764.211791:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24100864, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:1.0:1789760764.211808:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.211833:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760764.211838:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.211844:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.211867:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.221925:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760764.241747:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973201 is committed 00002000:00010000:2.0:1789760764.241963:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24100864 -> 24104960 00010000:00010000:2.0:1789760764.241980:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.241994:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.242003:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.242016:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760764.242021:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.242030:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.242049:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.242056:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f34e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.242177:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30466us (30680us total) trans 4294973201 rc 0/0 00000100:00100000:2.0:1789760764.242188:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5913 00000100:00100000:2.0:1789760764.242197:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742629467839 time 742676984418 state 0x2 ref 0 00000100:00100000:3.0:1789760764.253946:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760764.254091:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753728 00000100:00100000:2.0:1789760764.254101:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753728 00000100:00100000:2.0:1789760764.254153:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742629467839 time 742676984418 state 0x0 ref 1 00000100:00100000:2.0:1789760764.254168:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742677244834 time 742676984418 state 0x1 ref 1 00000100:00100000:2.0:1789760764.254198:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742677244834 time 742719487740 state 0x0 ref 1 00000100:00100000:2.0:1789760764.254214:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760764.254543:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24104960, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:2.0:1789760764.254552:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.254567:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760764.254571:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.254578:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.254592:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.305110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973202 is committed 00002000:00010000:3.0:1789760764.305212:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24104960 -> 24109056 00010000:00010000:3.0:1789760764.305228:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.305241:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.305251:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.305263:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.305268:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.305276:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.305295:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.305302:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f355 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.305425:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51208us (51482us total) trans 4294973202 rc 0/0 00000100:00100000:3.0:1789760764.305436:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5914 00000100:00100000:3.0:1789760764.305447:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742677244834 time 742719487740 state 0x2 ref 0 00000100:00100000:1.0:1789760764.309047:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760764.309141:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753856 00000100:00100000:2.0:1789760764.309151:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753856 00000100:00100000:2.0:1789760764.309201:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742677244834 time 742719487740 state 0x0 ref 1 00000100:00100000:2.0:1789760764.309215:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742719748156 time 742719487740 state 0x1 ref 1 00000100:00100000:2.0:1789760764.309284:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742719748156 time 742774572405 state 0x0 ref 1 00000100:00100000:2.0:1789760764.309296:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515753856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760764.309343:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24109056, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:2.0:1789760764.309352:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.309372:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760764.309377:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.309384:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.309397:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.369863:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24109056 -> 24113152 00010000:00010000:3.0:1789760764.369883:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.369896:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.369910:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.369917:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.369922:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.369931:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.369951:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.369958:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f35c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.370599:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515753856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61299us (61555us total) trans 4294973203 rc 0/0 00000100:00100000:3.0:1789760764.370611:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5915 00000100:00100000:3.0:1789760764.370621:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742719748156 time 742774572405 state 0x2 ref 0 00000001:00080000:1.0:1789760764.373120:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973203 is committed 00000100:00100000:0.0:1789760764.381494:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760764.381605:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515753984 00000100:00100000:3.0:1789760764.381615:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515753984 00000100:00100000:3.0:1789760764.381674:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742719748156 time 742774572405 state 0x0 ref 1 00000100:00100000:3.0:1789760764.381690:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742774832821 time 742774572405 state 0x1 ref 1 00000100:00100000:3.0:1789760764.381746:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742774832821 time 742847034891 state 0x0 ref 1 00000100:00100000:3.0:1789760764.381760:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515753984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.381809:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24113152, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:3.0:1789760764.381827:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.381843:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760764.381848:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.381854:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.381868:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.416585:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544416768:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760764.416650:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760764.417041:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416768 00000100:00100000:1.0:1789760764.417049:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416768 00000100:00100000:1.0:1789760764.417455:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789760764.417467:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544416768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760764.417499:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760764.417549:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544416768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (897us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760764.417556:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00080000:1.0:1789760764.417575:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760764.417576:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544416768:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760764.419457:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c029c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544416896:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760764.419523:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760764.419583:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544416896 00000100:00100000:2.0:1789760764.419590:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544416896 00000100:00100000:2.0:1789760764.419640:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:2.0:1789760764.419648:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544416896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760764.419670:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3834642432 avail 3813670912 00000100:00100000:2.0:1789760764.419742:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544416896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760764.419749:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:2.0:1789760764.419792:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c029c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544416896:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789760764.420760:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760764.433752:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973204 is committed 00002000:00010000:3.0:1789760764.433975:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24113152 -> 24117248 00010000:00010000:3.0:1789760764.433993:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.434006:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.434015:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.434022:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.434028:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.434036:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.434055:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.434062:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f363 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.434376:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515753984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52613us (52886us total) trans 4294973204 rc 0/0 00000100:00100000:3.0:1789760764.434387:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5916 00000100:00100000:3.0:1789760764.434397:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742774832821 time 742847034891 state 0x2 ref 0 00000100:00100000:0.0:1789760764.443951:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760764.444214:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754112 00000100:00100000:1.0:1789760764.444224:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754112 00000100:00100000:1.0:1789760764.444272:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742774832821 time 742847034891 state 0x0 ref 1 00000100:00100000:1.0:1789760764.444283:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742847295307 time 742847034891 state 0x1 ref 1 00000100:00100000:1.0:1789760764.444307:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742847295307 time 742909596485 state 0x0 ref 1 00000100:00100000:1.0:1789760764.444320:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515754112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760764.444375:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24117248, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb807 00010000:00010000:1.0:1789760764.444382:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.444395:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760764.444399:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.444403:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.444416:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.486672:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973205 is committed 00002000:00010000:0.0:1789760764.486855:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24117248 -> 24121344 00002000:00010000:0.0:1789760764.486861:0:13072:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:0.0:1789760764.486876:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.486889:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.486898:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.486912:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.486916:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.486925:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.486944:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.486953:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f36a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.487045:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515754112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42723us (43100us total) trans 4294973205 rc 0/0 00000100:00100000:0.0:1789760764.487056:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5917 00000100:00100000:0.0:1789760764.487066:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742847295307 time 742909596485 state 0x2 ref 0 00000100:00100000:2.0:1789760764.506999:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760764.507089:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754240 00000100:00100000:0.0:1789760764.507100:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754240 00000100:00100000:0.0:1789760764.507618:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742847295307 time 742909596485 state 0x0 ref 1 00000100:00100000:0.0:1789760764.507635:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742909856901 time 742909596485 state 0x1 ref 1 00000100:00100000:0.0:1789760764.507856:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742909856901 time 742973143115 state 0x0 ref 1 00000100:00100000:0.0:1789760764.507876:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515754240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760764.508213:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24121344, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:0.0:1789760764.508223:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.508239:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760764.508244:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.508251:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.508265:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.553963:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973206 is committed 00002000:00010000:3.0:1789760764.553996:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24121344 -> 24125440 00010000:00010000:3.0:1789760764.554011:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.554024:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.554033:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.554045:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.554051:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.554060:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.554083:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.554091:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f371 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.554182:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515754240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46306us (47187us total) trans 4294973206 rc 0/0 00000100:00100000:3.0:1789760764.554192:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5918 00000100:00100000:3.0:1789760764.554202:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742909856901 time 742973143115 state 0x2 ref 0 00000100:00100000:2.0:1789760764.560920:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760764.561076:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754368 00000100:00100000:0.0:1789760764.561086:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754368 00000100:00100000:0.0:1789760764.561141:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742909856901 time 742973143115 state 0x0 ref 1 00000100:00100000:0.0:1789760764.561156:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742973403531 time 742973143115 state 0x1 ref 1 00000100:00100000:0.0:1789760764.561185:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742973403531 time 743026474632 state 0x0 ref 1 00000100:00100000:0.0:1789760764.561197:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515754368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760764.561247:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24125440, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:0.0:1789760764.561254:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.561269:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.561273:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.561280:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.561294:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.586965:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973207 is committed 00002000:00010000:0.0:1789760764.587070:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24125440 -> 24129536 00010000:00010000:0.0:1789760764.587085:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.587099:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.587109:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.587116:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.587121:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.587131:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.587152:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.587161:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f378 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.587252:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515754368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26052us (26349us total) trans 4294973207 rc 0/0 00000100:00100000:0.0:1789760764.587263:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5919 00000100:00100000:0.0:1789760764.587273:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742973403531 time 743026474632 state 0x2 ref 0 00000100:00100000:0.0:1789760764.591669:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760764.591780:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754496 00000100:00100000:3.0:1789760764.591790:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754496 00000100:00100000:3.0:1789760764.591856:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 742973403531 time 743026474632 state 0x0 ref 1 00000100:00100000:3.0:1789760764.591871:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743026735048 time 743026474632 state 0x1 ref 1 00000100:00100000:3.0:1789760764.591900:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743026735048 time 743057188795 state 0x0 ref 1 00000100:00100000:3.0:1789760764.591913:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515754496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.591959:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24129536, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:3.0:1789760764.591968:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.591984:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.591988:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.591995:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.592010:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.627424:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24129536 -> 24133632 00010000:00010000:3.0:1789760764.627444:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.627470:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.627478:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.627485:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.627489:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.627498:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.627518:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.627526:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f37f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.627619:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515754496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35703us (35955us total) trans 4294973208 rc 0/0 00000100:00100000:3.0:1789760764.627631:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5920 00000100:00100000:3.0:1789760764.627641:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743026735048 time 743057188795 state 0x2 ref 0 00000001:00080000:1.0:1789760764.628917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973208 is committed 00000100:00100000:3.0:1789760764.632768:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760764.632875:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754624 00000100:00100000:2.0:1789760764.632885:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754624 00000100:00100000:2.0:1789760764.632938:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743026735048 time 743057188795 state 0x0 ref 1 00000100:00100000:2.0:1789760764.632957:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743057449211 time 743057188795 state 0x1 ref 1 00000100:00100000:2.0:1789760764.632986:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743057449211 time 743098275438 state 0x0 ref 1 00000100:00100000:2.0:1789760764.632999:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515754624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760764.633044:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24133632, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:2.0:1789760764.633054:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.633070:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.633075:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.633081:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.633094:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760764.671486:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24133632 -> 24137728 00010000:00010000:2.0:1789760764.671507:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.671522:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.671531:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.671544:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760764.671549:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.671559:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.671579:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.671588:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f386 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.671685:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515754624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38684us (38974us total) trans 4294973209 rc 0/0 00000100:00100000:2.0:1789760764.671696:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5921 00000100:00100000:2.0:1789760764.671746:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743057449211 time 743098275438 state 0x2 ref 0 00000001:00080000:0.0:1789760764.672278:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973209 is committed 00000100:00100000:2.0:1789760764.682839:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760764.683020:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754752 00000100:00100000:3.0:1789760764.683029:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754752 00000100:00100000:3.0:1789760764.683081:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743057449211 time 743098275438 state 0x0 ref 1 00000100:00100000:3.0:1789760764.683097:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743098535854 time 743098275438 state 0x1 ref 1 00000100:00100000:3.0:1789760764.683130:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743098535854 time 743148419346 state 0x0 ref 1 00000100:00100000:3.0:1789760764.683143:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515754752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.683188:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24137728, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:3.0:1789760764.683197:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.683212:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.683217:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.683223:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.683235:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.754256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973210 is committed 00002000:00010000:0.0:1789760764.754345:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24137728 -> 24141824 00010000:00010000:0.0:1789760764.754362:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.754375:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.754384:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.754391:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.754396:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.754405:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.754427:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.754435:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f38d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.754566:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515754752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 71420us (71732us total) trans 4294973210 rc 0/0 00000100:00100000:0.0:1789760764.754577:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5922 00000100:00100000:0.0:1789760764.754586:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743098535854 time 743148419346 state 0x2 ref 0 00000100:00100000:1.0:1789760764.758647:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760764.758865:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515754880 00000100:00100000:0.0:1789760764.758875:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515754880 00000100:00100000:0.0:1789760764.758928:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743098535854 time 743148419346 state 0x0 ref 1 00000100:00100000:0.0:1789760764.758945:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743148679762 time 743148419346 state 0x1 ref 1 00000100:00100000:0.0:1789760764.758968:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743148679762 time 743224257616 state 0x0 ref 1 00000100:00100000:0.0:1789760764.758979:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515754880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760764.759027:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24141824, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:0.0:1789760764.759037:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.759053:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.759057:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.759063:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.759077:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.792527:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973211 is committed 00002000:00010000:0.0:1789760764.792588:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24141824 -> 24145920 00010000:00010000:0.0:1789760764.792603:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.792616:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.792626:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.792634:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.792639:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.792647:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.792667:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.792681:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f394 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.792864:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515754880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33882us (34222us total) trans 4294973211 rc 0/0 00000100:00100000:0.0:1789760764.792876:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5923 00000100:00100000:0.0:1789760764.792885:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743148679762 time 743224257616 state 0x2 ref 0 00000100:00100000:2.0:1789760764.797266:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760764.797611:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755008 00000100:00100000:3.0:1789760764.797623:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755008 00000100:00100000:3.0:1789760764.797683:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743148679762 time 743224257616 state 0x0 ref 1 00000100:00100000:3.0:1789760764.797699:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743224518032 time 743224257616 state 0x1 ref 1 00000100:00100000:3.0:1789760764.797819:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743224518032 time 743263097284 state 0x0 ref 1 00000100:00100000:3.0:1789760764.797833:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515755008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.797892:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24145920, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:3.0:1789760764.797901:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.797917:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.797922:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.797929:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.797945:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.826568:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24145920 -> 24150016 00010000:00010000:0.0:1789760764.826590:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.826604:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.826614:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.826622:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.826627:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.826636:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.826662:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.826670:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f39b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.826800:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515755008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28964us (29539us total) trans 4294973212 rc 0/0 00000100:00100000:0.0:1789760764.826820:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5924 00000100:00100000:0.0:1789760764.826831:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743224518032 time 743263097284 state 0x2 ref 0 00000001:00080000:1.0:1789760764.827215:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973212 is committed 00000100:00100000:0.0:1789760764.830638:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760764.831188:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755136 00000100:00100000:2.0:1789760764.831199:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755136 00000100:00100000:2.0:1789760764.831254:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743224518032 time 743263097284 state 0x0 ref 1 00000100:00100000:2.0:1789760764.831268:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743263357700 time 743263097284 state 0x1 ref 1 00000100:00100000:2.0:1789760764.831300:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743263357700 time 743296589157 state 0x0 ref 1 00000100:00100000:2.0:1789760764.831312:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515755136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760764.831360:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24150016, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc007 00010000:00010000:2.0:1789760764.831369:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.831385:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.831390:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.831396:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.831408:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.862049:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973213 is committed 00002000:00010000:3.0:1789760764.862138:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24150016 -> 24154112 00002000:00010000:3.0:1789760764.862143:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760764.862146:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00010000:00010000:3.0:1789760764.862161:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.862175:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.862185:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.862192:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.862197:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.862206:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.862225:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.862239:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f3a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.862365:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515755136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31049us (31730us total) trans 4294973213 rc 0/0 00000100:00100000:3.0:1789760764.862375:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5925 00000100:00100000:3.0:1789760764.862385:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743263357700 time 743296589157 state 0x2 ref 0 00000100:00100000:0.0:1789760764.866703:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760764.866969:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755264 00000100:00100000:1.0:1789760764.866978:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755264 00000100:00100000:1.0:1789760764.867033:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743263357700 time 743296589157 state 0x0 ref 1 00000100:00100000:1.0:1789760764.867047:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743296849573 time 743296589157 state 0x1 ref 1 00000100:00100000:1.0:1789760764.867112:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743296849573 time 743332401380 state 0x0 ref 1 00000100:00100000:1.0:1789760764.867126:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760764.867178:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24154112, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:1.0:1789760764.867187:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.867202:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.867206:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.867213:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.867227:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.898824:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973214 is committed 00002000:00010000:3.0:1789760764.898997:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24154112 -> 24158208 00010000:00010000:3.0:1789760764.899013:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.899026:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.899034:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.899042:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.899047:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.899057:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.899081:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.899088:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f3a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.899211:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32082us (32512us total) trans 4294973214 rc 0/0 00000100:00100000:3.0:1789760764.899221:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5926 00000100:00100000:3.0:1789760764.899231:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743296849573 time 743332401380 state 0x2 ref 0 00000100:00100000:0.0:1789760764.902264:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760764.902336:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755392 00000100:00100000:3.0:1789760764.902345:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755392 00000100:00100000:3.0:1789760764.902394:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743296849573 time 743332401380 state 0x0 ref 1 00000100:00100000:3.0:1789760764.902409:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743332661796 time 743332401380 state 0x1 ref 1 00000100:00100000:3.0:1789760764.902438:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743332661796 time 743367727220 state 0x0 ref 1 00000100:00100000:3.0:1789760764.902449:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.902603:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24158208, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760764.902612:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.902628:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.902633:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.902640:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.902654:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.935436:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973215 is committed 00002000:00010000:0.0:1789760764.935518:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24158208 -> 24162304 00010000:00010000:0.0:1789760764.935533:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.935546:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.935557:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.935571:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.935576:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.935585:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.935607:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.935615:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f3b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.935875:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33255us (33446us total) trans 4294973215 rc 0/0 00000100:00100000:0.0:1789760764.935887:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5927 00000100:00100000:0.0:1789760764.935897:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743332661796 time 743367727220 state 0x2 ref 0 00000100:00100000:0.0:1789760764.939529:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760764.939620:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755520 00000100:00100000:3.0:1789760764.939630:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755520 00000100:00100000:3.0:1789760764.939683:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743332661796 time 743367727220 state 0x0 ref 1 00000100:00100000:3.0:1789760764.939699:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743367987636 time 743367727220 state 0x1 ref 1 00000100:00100000:3.0:1789760764.939771:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743367987636 time 743405059313 state 0x0 ref 1 00000100:00100000:3.0:1789760764.939783:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760764.939839:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24162304, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760764.939848:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.939863:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.939867:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.939873:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.939887:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.964856:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973216 is committed 00002000:00010000:3.0:1789760764.965003:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24162304 -> 24166400 00010000:00010000:3.0:1789760764.965020:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.965034:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.965043:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.965050:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760764.965055:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.965064:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.965082:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.965091:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.965219:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25432us (25693us total) trans 4294973216 rc 0/0 00000100:00100000:3.0:1789760764.965233:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5928 00000100:00100000:3.0:1789760764.965243:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743367987636 time 743405059313 state 0x2 ref 0 00000100:00100000:1.0:1789760764.968368:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760764.968526:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755648 00000100:00100000:0.0:1789760764.968536:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755648 00000100:00100000:0.0:1789760764.968587:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743367987636 time 743405059313 state 0x0 ref 1 00000100:00100000:0.0:1789760764.968602:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743405319729 time 743405059313 state 0x1 ref 1 00000100:00100000:0.0:1789760764.968661:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743405319729 time 743433949830 state 0x0 ref 1 00000100:00100000:0.0:1789760764.968674:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515755648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760764.968797:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24166400, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:0.0:1789760764.968814:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.968830:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.968834:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.968840:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.968854:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.994457:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973217 is committed 00002000:00010000:0.0:1789760764.994543:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24166400 -> 24170496 00010000:00010000:0.0:1789760764.994558:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.994570:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.994579:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.994586:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760764.994590:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.994599:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.994619:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.994626:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.994862:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515755648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26184us (26498us total) trans 4294973217 rc 0/0 00000100:00100000:0.0:1789760764.994873:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5929 00000100:00100000:0.0:1789760764.994883:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743405319729 time 743433949830 state 0x2 ref 0 00000100:00100000:0.0:1789760765.003197:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.003286:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755776 00000100:00100000:3.0:1789760765.003296:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755776 00000100:00100000:3.0:1789760765.003350:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743405319729 time 743433949830 state 0x0 ref 1 00000100:00100000:3.0:1789760765.003365:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743434210246 time 743433949830 state 0x1 ref 1 00000100:00100000:3.0:1789760765.003397:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743434210246 time 743468686439 state 0x0 ref 1 00000100:00100000:3.0:1789760765.003409:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515755776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.003466:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24170496, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.003475:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.003492:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.003496:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.003503:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.003516:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.034414:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24170496 -> 24174592 00000001:00080000:0.0:1789760765.034423:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973218 is committed 00010000:00010000:3.0:1789760765.034433:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.034446:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.034456:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.034463:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.034468:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.034476:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.034494:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.034501:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.034594:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515755776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31178us (31398us total) trans 4294973218 rc 0/0 00000100:00100000:3.0:1789760765.034606:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5930 00000100:00100000:3.0:1789760765.034615:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743434210246 time 743468686439 state 0x2 ref 0 00000100:00100000:2.0:1789760765.038015:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760765.038111:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515755904 00000100:00100000:0.0:1789760765.038120:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515755904 00000100:00100000:0.0:1789760765.038171:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743434210246 time 743468686439 state 0x0 ref 1 00000100:00100000:0.0:1789760765.038186:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743468946855 time 743468686439 state 0x1 ref 1 00000100:00100000:0.0:1789760765.038238:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743468946855 time 743503527661 state 0x0 ref 1 00000100:00100000:0.0:1789760765.038251:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760765.038300:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24174592, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:0.0:1789760765.038309:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.038325:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760765.038329:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.038335:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.038349:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.070698:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973219 is committed 00002000:00010000:0.0:1789760765.071018:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24174592 -> 24178688 00010000:00010000:0.0:1789760765.071036:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.071051:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.071062:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.071071:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.071077:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.071087:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.071109:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.071118:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.071209:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515755904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32955us (33197us total) trans 4294973219 rc 0/0 00000100:00100000:0.0:1789760765.071218:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5931 00000100:00100000:0.0:1789760765.071228:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743468946855 time 743503527661 state 0x2 ref 0 00000100:00100000:1.0:1789760765.074869:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.074954:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756032 00000100:00100000:3.0:1789760765.074963:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756032 00000100:00100000:3.0:1789760765.075020:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743468946855 time 743503527661 state 0x0 ref 1 00000100:00100000:3.0:1789760765.075035:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743503788077 time 743503527661 state 0x1 ref 1 00000100:00100000:3.0:1789760765.075065:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743503788077 time 743540353896 state 0x0 ref 1 00000100:00100000:3.0:1789760765.075077:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515756032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.075123:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24178688, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.075131:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.075146:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.075151:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.075158:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.075171:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.098875:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973220 is committed 00002000:00010000:3.0:1789760765.098947:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24178688 -> 24182784 00010000:00010000:3.0:1789760765.098962:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.098976:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.098986:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.098995:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.098999:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.099008:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.099027:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.099035:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.099116:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515756032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24037us (24252us total) trans 4294973220 rc 0/0 00000100:00100000:3.0:1789760765.099125:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5932 00000100:00100000:3.0:1789760765.099134:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743503788077 time 743540353896 state 0x2 ref 0 00000100:00100000:2.0:1789760765.102326:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760765.102755:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756160 00000100:00100000:0.0:1789760765.102766:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756160 00000100:00100000:0.0:1789760765.102825:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743503788077 time 743540353896 state 0x0 ref 1 00000100:00100000:0.0:1789760765.102841:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743540614312 time 743540353896 state 0x1 ref 1 00000100:00100000:0.0:1789760765.102864:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743540614312 time 743568153641 state 0x0 ref 1 00000100:00100000:0.0:1789760765.102877:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515756160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760765.102924:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24182784, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:0.0:1789760765.102933:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.102948:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760765.102953:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.102959:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.102972:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.129001:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24182784 -> 24186880 00010000:00010000:0.0:1789760765.129021:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.129035:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.129045:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.129052:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.129058:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.129067:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.129087:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.129100:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.129194:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515756160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26315us (26873us total) trans 4294973221 rc 0/0 00000100:00100000:0.0:1789760765.129204:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5933 00000100:00100000:0.0:1789760765.129214:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743540614312 time 743568153641 state 0x2 ref 0 00000001:00080000:2.0:1789760765.131565:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973221 is committed 00000100:00100000:3.0:1789760765.132918:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760765.133232:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756288 00000100:00100000:0.0:1789760765.133242:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756288 00000100:00100000:0.0:1789760765.133294:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743540614312 time 743568153641 state 0x0 ref 1 00000100:00100000:0.0:1789760765.133309:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743568414057 time 743568153641 state 0x1 ref 1 00000100:00100000:0.0:1789760765.133337:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743568414057 time 743598626711 state 0x0 ref 1 00000100:00100000:0.0:1789760765.133349:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760765.133392:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24186880, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:0.0:1789760765.133401:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.133416:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760765.133420:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.133427:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.133446:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.165383:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973222 is committed 00002000:00010000:3.0:1789760765.165546:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24186880 -> 24190976 00010000:00010000:3.0:1789760765.165561:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.165579:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.165588:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.165595:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.165600:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.165610:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.165629:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.165638:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f3e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.165937:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32583us (33021us total) trans 4294973222 rc 0/0 00000100:00100000:3.0:1789760765.165950:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5934 00000100:00100000:3.0:1789760765.165961:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743568414057 time 743598626711 state 0x2 ref 0 00000100:00100000:1.0:1789760765.169205:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760765.169302:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756416 00000100:00100000:0.0:1789760765.169313:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756416 00000100:00100000:0.0:1789760765.169361:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743568414057 time 743598626711 state 0x0 ref 1 00000100:00100000:0.0:1789760765.169376:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743598887127 time 743598626711 state 0x1 ref 1 00000100:00100000:0.0:1789760765.169435:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743598887127 time 743634723699 state 0x0 ref 1 00000100:00100000:0.0:1789760765.169517:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760765.169564:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24190976, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:0.0:1789760765.169572:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.169589:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760765.169594:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.169600:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.169614:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.197574:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973223 is committed 00002000:00010000:1.0:1789760765.197925:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24190976 -> 24195072 00010000:00010000:1.0:1789760765.197941:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.197955:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.197964:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.197972:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.197977:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.197987:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.198015:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.198023:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f3e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.198123:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28670us (28921us total) trans 4294973223 rc 0/0 00000100:00100000:1.0:1789760765.198134:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5935 00000100:00100000:1.0:1789760765.198144:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743598887127 time 743634723699 state 0x2 ref 0 00000100:00100000:2.0:1789760765.202225:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.202318:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756544 00000100:00100000:3.0:1789760765.202328:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756544 00000100:00100000:3.0:1789760765.202378:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743598887127 time 743634723699 state 0x0 ref 1 00000100:00100000:3.0:1789760765.202394:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743634984115 time 743634723699 state 0x1 ref 1 00000100:00100000:3.0:1789760765.202428:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743634984115 time 743667717092 state 0x0 ref 1 00000100:00100000:3.0:1789760765.202439:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515756544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.202497:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24195072, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.202506:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.202522:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.202526:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.202533:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.202545:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.243038:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973224 is committed 00002000:00010000:0.0:1789760765.243126:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24195072 -> 24199168 00010000:00010000:0.0:1789760765.243143:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.243156:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.243166:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.243173:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.243178:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.243188:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.243208:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.243217:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1f3ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.243371:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515756544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40928us (41150us total) trans 4294973224 rc 0/0 00000100:00100000:0.0:1789760765.243391:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5936 00000100:00100000:0.0:1789760765.243401:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743634984115 time 743667717092 state 0x2 ref 0 00010000:00010000:3.0:1789760765.247233:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760765.247371:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.247513:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756672 00000100:00100000:3.0:1789760765.247522:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756672 00000100:00100000:3.0:1789760765.247571:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743634984115 time 743667717092 state 0x0 ref 1 00000100:00100000:3.0:1789760765.247586:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743667977508 time 743667717092 state 0x1 ref 1 00000100:00100000:3.0:1789760765.247644:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743667977508 time 743712932553 state 0x0 ref 1 00000100:00100000:3.0:1789760765.247658:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.247703:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24199168, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.247743:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.247759:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.247763:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.247769:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.247783:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.270915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973225 is committed 00002000:00010000:1.0:1789760765.271014:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24199168 -> 24203264 00010000:00010000:1.0:1789760765.271030:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.271040:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.271046:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.271051:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.271054:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.271064:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.271078:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.271083:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f3f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.271180:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23522us (23811us total) trans 4294973225 rc 0/0 00000100:00100000:1.0:1789760765.271188:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5937 00000100:00100000:1.0:1789760765.271195:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743667977508 time 743712932553 state 0x2 ref 0 00000100:00100000:2.0:1789760765.274309:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.274465:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756800 00000100:00100000:3.0:1789760765.274476:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756800 00000100:00100000:3.0:1789760765.274529:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743667977508 time 743712932553 state 0x0 ref 1 00000100:00100000:3.0:1789760765.274544:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743713192969 time 743712932553 state 0x1 ref 1 00000100:00100000:3.0:1789760765.274611:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743713192969 time 743739899483 state 0x0 ref 1 00000100:00100000:3.0:1789760765.274624:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515756800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.274672:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24203264, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.274681:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.274702:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.274813:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.274821:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.274835:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.299242:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973226 is committed 00002000:00010000:3.0:1789760765.299481:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24203264 -> 24207360 00010000:00010000:3.0:1789760765.299498:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.299512:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.299522:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.299530:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.299535:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.299545:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.299566:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.299576:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f3fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.299703:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515756800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25075us (25397us total) trans 4294973226 rc 0/0 00000100:00100000:3.0:1789760765.299826:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5938 00000100:00100000:3.0:1789760765.299836:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743713192969 time 743739899483 state 0x2 ref 0 00000100:00100000:2.0:1789760765.312352:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.312513:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515756928 00000100:00100000:3.0:1789760765.312522:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515756928 00000100:00100000:3.0:1789760765.312571:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743713192969 time 743739899483 state 0x0 ref 1 00000100:00100000:3.0:1789760765.312585:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743740159899 time 743739899483 state 0x1 ref 1 00000100:00100000:3.0:1789760765.312617:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743740159899 time 743777906153 state 0x0 ref 1 00000100:00100000:3.0:1789760765.312630:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515756928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.312677:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24207360, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.312686:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.312702:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.312731:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.312738:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.312758:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760765.341415:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24207360 -> 24211456 00010000:00010000:1.0:1789760765.341435:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.341448:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.341458:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.341465:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.341470:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.341479:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.341498:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.341506:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f404 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.341598:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515756928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28965us (29250us total) trans 4294973227 rc 0/0 00000100:00100000:1.0:1789760765.341610:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5939 00000100:00100000:1.0:1789760765.341621:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743740159899 time 743777906153 state 0x2 ref 0 00000001:00080000:2.0:1789760765.342268:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973227 is committed 00000100:00100000:2.0:1789760765.346665:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760765.346915:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757056 00000100:00100000:0.0:1789760765.346925:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757056 00000100:00100000:3.0:1789760765.347838:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743740159899 time 743777906153 state 0x0 ref 1 00000100:00100000:3.0:1789760765.347855:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743778166569 time 743777906153 state 0x1 ref 1 00000100:00100000:3.0:1789760765.347889:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743778166569 time 743813178553 state 0x0 ref 1 00000100:00100000:3.0:1789760765.347902:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515757056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.347950:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24211456, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.347960:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.347976:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760765.347981:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.347987:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.348000:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.377915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973228 is committed 00002000:00010000:0.0:1789760765.378015:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24211456 -> 24215552 00010000:00010000:0.0:1789760765.378030:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.378043:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.378053:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.378062:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.378067:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.378077:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.378106:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.378114:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f40b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.378206:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515757056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30301us (31545us total) trans 4294973228 rc 0/0 00000100:00100000:0.0:1789760765.378219:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5940 00000100:00100000:0.0:1789760765.378228:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743778166569 time 743813178553 state 0x2 ref 0 00000100:00100000:3.0:1789760765.389410:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.394340:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757184 00000100:00100000:1.0:1789760765.394356:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757184 00000100:00100000:1.0:1789760765.394420:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743778166569 time 743813178553 state 0x0 ref 1 00000100:00100000:1.0:1789760765.394434:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743813438969 time 743813178553 state 0x1 ref 1 00000100:00100000:1.0:1789760765.394576:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743813438969 time 743859863917 state 0x0 ref 1 00000100:00100000:1.0:1789760765.394590:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760765.394635:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24215552, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:1.0:1789760765.394644:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.394659:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760765.394663:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.394669:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.394681:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760765.428918:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24215552 -> 24219648 00010000:00010000:1.0:1789760765.428937:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.428949:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.428958:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.428965:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.428969:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.428977:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.429003:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.429011:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f412 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.429109:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515757184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34510us (39695us total) trans 4294973229 rc 0/0 00000100:00100000:1.0:1789760765.429119:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5941 00000100:00100000:1.0:1789760765.429129:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743813438969 time 743859863917 state 0x2 ref 0 00000001:00080000:0.0:1789760765.430085:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973229 is committed 00000100:00100000:2.0:1789760765.433694:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760765.436360:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757312 00000100:00100000:0.0:1789760765.436374:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757312 00000100:00100000:0.0:1789760765.436433:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743813438969 time 743859863917 state 0x0 ref 1 00000100:00100000:0.0:1789760765.436457:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743860124333 time 743859863917 state 0x1 ref 1 00000100:00100000:0.0:1789760765.436485:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743860124333 time 743901774773 state 0x0 ref 1 00000100:00100000:0.0:1789760765.436496:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760765.436542:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24219648, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:0.0:1789760765.436551:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.436566:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760765.436571:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.436576:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.436589:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.484763:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973230 is committed 00002000:00010000:1.0:1789760765.484928:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24219648 -> 24223744 00010000:00010000:1.0:1789760765.484945:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.484958:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.484967:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.484973:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.484978:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.484993:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.485012:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.485020:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f419 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.485175:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48675us (51484us total) trans 4294973230 rc 0/0 00000100:00100000:1.0:1789760765.485186:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5942 00000100:00100000:1.0:1789760765.485196:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743860124333 time 743901774773 state 0x2 ref 0 00000100:00100000:2.0:1789760765.487910:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.487970:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757440 00000100:00100000:1.0:1789760765.487980:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757440 00000100:00100000:1.0:1789760765.488034:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743860124333 time 743901774773 state 0x0 ref 1 00000100:00100000:1.0:1789760765.488050:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743902035189 time 743901774773 state 0x1 ref 1 00000100:00100000:1.0:1789760765.488081:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743902035189 time 743953370613 state 0x0 ref 1 00000100:00100000:1.0:1789760765.488094:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760765.488152:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24223744, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:1.0:1789760765.488161:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.488179:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760765.488183:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.488189:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.488201:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.520372:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973231 is committed 00002000:00010000:1.0:1789760765.520418:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24223744 -> 24227840 00010000:00010000:1.0:1789760765.520433:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.520445:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.520455:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.520462:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.520467:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.520476:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.520496:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.520505:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f420 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.520864:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32766us (32957us total) trans 4294973231 rc 0/0 00000100:00100000:1.0:1789760765.520875:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5943 00000100:00100000:1.0:1789760765.520886:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743902035189 time 743953370613 state 0x2 ref 0 00000100:00100000:3.0:1789760765.530380:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760765.530910:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757568 00000100:00100000:2.0:1789760765.530921:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757568 00000100:00100000:2.0:1789760765.530970:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743902035189 time 743953370613 state 0x0 ref 1 00000100:00100000:2.0:1789760765.530984:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743953631029 time 743953370613 state 0x1 ref 1 00000100:00100000:2.0:1789760765.531014:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743953631029 time 743996303602 state 0x0 ref 1 00000100:00100000:2.0:1789760765.531025:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760765.531072:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24227840, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:2.0:1789760765.531081:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.531104:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760765.531108:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.531116:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.531130:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.594118:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24227840 -> 24231936 00010000:00010000:2.0:1789760765.594137:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.594151:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.594160:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.594167:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760765.594172:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.594180:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.594201:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.594209:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f427 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.594303:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515757568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63273us (63927us total) trans 4294973232 rc 0/0 00000100:00100000:2.0:1789760765.594316:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5944 00000100:00100000:2.0:1789760765.594326:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743953631029 time 743996303602 state 0x2 ref 0 00000001:00080000:2.0:1789760765.594384:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973232 is committed 00000100:00100000:2.0:1789760765.603957:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.609059:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757696 00000100:00100000:1.0:1789760765.609075:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757696 00000100:00100000:2.0:1789760765.612268:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743953631029 time 743996303602 state 0x0 ref 1 00000100:00100000:2.0:1789760765.612287:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743996564018 time 743996303602 state 0x1 ref 1 00000100:00100000:2.0:1789760765.612378:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743996564018 time 744077665860 state 0x0 ref 1 00000100:00100000:2.0:1789760765.612394:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760765.612446:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24231936, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:2.0:1789760765.612455:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.612470:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.612475:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.612481:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.612495:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.679361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973233 is committed 00002000:00010000:0.0:1789760765.679464:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24231936 -> 24236032 00010000:00010000:0.0:1789760765.679492:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.679506:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.679515:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.679522:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.679527:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.679536:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.679556:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.679563:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f42e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.679657:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67262us (75706us total) trans 4294973233 rc 0/0 00000100:00100000:0.0:1789760765.679668:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5945 00000100:00100000:0.0:1789760765.679677:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743996564018 time 744077665860 state 0x2 ref 0 00000100:00100000:0.0:1789760765.684028:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.684139:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757824 00000100:00100000:3.0:1789760765.684150:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757824 00000100:00100000:3.0:1789760765.684204:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 743996564018 time 744077665860 state 0x0 ref 1 00000100:00100000:3.0:1789760765.684219:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744077926276 time 744077665860 state 0x1 ref 1 00000100:00100000:3.0:1789760765.684288:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744077926276 time 744149576676 state 0x0 ref 1 00000100:00100000:3.0:1789760765.684302:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515757824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.684352:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24236032, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:3.0:1789760765.684361:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.684378:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.684383:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.684389:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.684403:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.729967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973234 is committed 00002000:00010000:1.0:1789760765.730133:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24236032 -> 24240128 00010000:00010000:1.0:1789760765.730150:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.730164:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.730173:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.730180:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760765.730184:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.730194:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.730218:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.730227:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f435 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.730323:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515757824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46019us (46301us total) trans 4294973234 rc 0/0 00000100:00100000:1.0:1789760765.730334:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5946 00000100:00100000:1.0:1789760765.730345:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744077926276 time 744149576676 state 0x2 ref 0 00000100:00100000:2.0:1789760765.735219:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.735303:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515757952 00000100:00100000:1.0:1789760765.735313:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515757952 00000100:00100000:1.0:1789760765.735364:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744077926276 time 744149576676 state 0x0 ref 1 00000100:00100000:1.0:1789760765.735378:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744149837092 time 744149576676 state 0x1 ref 1 00000100:00100000:1.0:1789760765.735433:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744149837092 time 744200721766 state 0x0 ref 1 00000100:00100000:1.0:1789760765.735511:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760765.735558:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24240128, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:1.0:1789760765.735566:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.735582:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.735587:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.735593:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.735609:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.763189:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973235 is committed 00002000:00010000:0.0:1789760765.763293:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24240128 -> 24244224 00010000:00010000:0.0:1789760765.763309:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.763322:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.763331:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.763339:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.763344:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.763353:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.763373:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.763381:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f43c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.763585:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515757952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28137us (28372us total) trans 4294973235 rc 0/0 00000100:00100000:0.0:1789760765.763596:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5947 00000100:00100000:0.0:1789760765.763607:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744149837092 time 744200721766 state 0x2 ref 0 00000100:00100000:2.0:1789760765.768508:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.768596:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758080 00000100:00100000:1.0:1789760765.768606:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758080 00000100:00100000:1.0:1789760765.768658:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744149837092 time 744200721766 state 0x0 ref 1 00000100:00100000:1.0:1789760765.768674:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744200982182 time 744200721766 state 0x1 ref 1 00000100:00100000:1.0:1789760765.768819:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744200982182 time 744234106423 state 0x0 ref 1 00000100:00100000:1.0:1789760765.768833:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515758080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760765.768945:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24244224, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:1.0:1789760765.768955:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.768971:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.768976:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.768983:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.768997:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.804310:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24244224 -> 24248320 00010000:00010000:0.0:1789760765.804332:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.804345:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.804355:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.804363:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760765.804368:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.804377:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.804398:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.804406:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f443 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.804589:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515758080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35753us (36137us total) trans 4294973236 rc 0/0 00000100:00100000:0.0:1789760765.804599:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5948 00000100:00100000:0.0:1789760765.804608:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744200982182 time 744234106423 state 0x2 ref 0 00000001:00080000:2.0:1789760765.804685:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973236 is committed 00000100:00100000:0.0:1789760765.808687:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.808993:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758208 00000100:00100000:1.0:1789760765.809004:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758208 00000100:00100000:1.0:1789760765.809055:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744200982182 time 744234106423 state 0x0 ref 1 00000100:00100000:1.0:1789760765.809069:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744234366839 time 744234106423 state 0x1 ref 1 00000100:00100000:1.0:1789760765.809137:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744234366839 time 744274425720 state 0x0 ref 1 00000100:00100000:1.0:1789760765.809150:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515758208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760765.809202:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24248320, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:1.0:1789760765.809210:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.809225:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.809229:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.809235:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.809247:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.841919:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973237 is committed 00002000:00010000:3.0:1789760765.842156:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24248320 -> 24252416 00010000:00010000:3.0:1789760765.842171:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.842184:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.842198:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.842206:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.842211:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.842219:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.842238:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.842246:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f44a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.842341:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515758208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33187us (33658us total) trans 4294973237 rc 0/0 00000100:00100000:3.0:1789760765.842353:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5949 00000100:00100000:3.0:1789760765.842362:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744234366839 time 744274425720 state 0x2 ref 0 00000100:00100000:1.0:1789760765.846592:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760765.847470:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758336 00000100:00100000:2.0:1789760765.847527:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758336 00000100:00100000:2.0:1789760765.847578:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744234366839 time 744274425720 state 0x0 ref 1 00000100:00100000:2.0:1789760765.847593:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744274686136 time 744274425720 state 0x1 ref 1 00000100:00100000:2.0:1789760765.847830:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744274686136 time 744313116868 state 0x0 ref 1 00000100:00100000:2.0:1789760765.847844:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515758336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760765.847893:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24252416, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc007 00010000:00010000:2.0:1789760765.847902:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847919:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.847923:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.847929:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.847944:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.879630:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24252416 -> 24256512 00002000:00010000:3.0:1789760765.879639:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:3.0:1789760765.879641:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00010000:00010000:3.0:1789760765.879653:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.879667:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.879677:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.879685:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.879690:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.879699:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.879741:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.879749:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f451 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.879844:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515758336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31998us (33255us total) trans 4294973238 rc 0/0 00000100:00100000:3.0:1789760765.879853:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5950 00000100:00100000:3.0:1789760765.879863:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744274686136 time 744313116868 state 0x2 ref 0 00000001:00080000:1.0:1789760765.881374:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973238 is committed 00000100:00100000:0.0:1789760765.883176:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.883455:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758464 00000100:00100000:3.0:1789760765.883467:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758464 00000100:00100000:3.0:1789760765.883521:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744274686136 time 744313116868 state 0x0 ref 1 00000100:00100000:3.0:1789760765.883537:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744313377284 time 744313116868 state 0x1 ref 1 00000100:00100000:3.0:1789760765.883568:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744313377284 time 744348857303 state 0x0 ref 1 00000100:00100000:3.0:1789760765.883580:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515758464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.883632:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24256512, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760765.883642:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.883658:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.883663:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.883670:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.883684:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.913792:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24256512 -> 24260608 00010000:00010000:3.0:1789760765.913829:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.913842:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.913855:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.913862:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.913867:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.913877:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.913895:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.913903:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f458 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.913988:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515758464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30405us (30816us total) trans 4294973239 rc 0/0 00000100:00100000:3.0:1789760765.913998:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5951 00000100:00100000:3.0:1789760765.914008:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744313377284 time 744348857303 state 0x2 ref 0 00000001:00080000:0.0:1789760765.916862:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973239 is committed 00000100:00100000:0.0:1789760765.919763:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760765.920100:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758592 00000100:00100000:3.0:1789760765.920109:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758592 00000100:00100000:3.0:1789760765.920160:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744313377284 time 744348857303 state 0x0 ref 1 00000100:00100000:3.0:1789760765.920174:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744349117719 time 744348857303 state 0x1 ref 1 00000100:00100000:3.0:1789760765.920208:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744349117719 time 744385496681 state 0x0 ref 1 00000100:00100000:3.0:1789760765.920220:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515758592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760765.920269:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24260608, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760765.920278:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.920294:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.920299:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.920305:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.920319:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.954384:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973240 is committed 00002000:00010000:3.0:1789760765.954564:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24260608 -> 24264704 00010000:00010000:3.0:1789760765.954582:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.954596:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.954606:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.954614:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760765.954619:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.954628:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.954648:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.954655:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f45f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.954892:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515758592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34669us (35133us total) trans 4294973240 rc 0/0 00000100:00100000:3.0:1789760765.954904:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5952 00000100:00100000:3.0:1789760765.954914:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744349117719 time 744385496681 state 0x2 ref 0 00000100:00100000:0.0:1789760765.958403:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760765.958568:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758720 00000100:00100000:1.0:1789760765.958577:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758720 00000100:00100000:1.0:1789760765.958630:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744349117719 time 744385496681 state 0x0 ref 1 00000100:00100000:1.0:1789760765.958645:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744385757097 time 744385496681 state 0x1 ref 1 00000100:00100000:1.0:1789760765.958734:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744385757097 time 744423993214 state 0x0 ref 1 00000100:00100000:1.0:1789760765.958748:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515758720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760765.958793:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24264704, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:1.0:1789760765.958811:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.958827:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.958831:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.958837:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.958851:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.990940:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24264704 -> 24268800 00010000:00010000:2.0:1789760765.990962:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.990978:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.990987:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.990995:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760765.990999:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.991008:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.991028:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.991036:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f466 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.991128:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515758720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32378us (32729us total) trans 4294973241 rc 0/0 00000100:00100000:2.0:1789760765.991139:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5953 00000100:00100000:2.0:1789760765.991148:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744385757097 time 744423993214 state 0x2 ref 0 00000001:00080000:3.0:1789760765.993786:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973241 is committed 00000100:00100000:0.0:1789760765.995628:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760765.995857:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758848 00000100:00100000:2.0:1789760765.995867:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758848 00000100:00100000:2.0:1789760765.995917:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744385757097 time 744423993214 state 0x0 ref 1 00000100:00100000:2.0:1789760765.995930:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744424253630 time 744423993214 state 0x1 ref 1 00000100:00100000:2.0:1789760765.995951:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744424253630 time 744461241196 state 0x0 ref 1 00000100:00100000:2.0:1789760765.995962:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515758848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760765.996011:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24268800, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760765.996020:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.996035:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.996039:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.996044:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.996056:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.026611:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973242 is committed 00002000:00010000:0.0:1789760766.026861:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24268800 -> 24272896 00010000:00010000:0.0:1789760766.026878:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.026890:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.026901:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.026908:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.026913:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.026926:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.026945:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.026954:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f46d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.027084:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515758848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31119us (31460us total) trans 4294973242 rc 0/0 00000100:00100000:0.0:1789760766.027095:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5954 00000100:00100000:0.0:1789760766.027105:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744424253630 time 744461241196 state 0x2 ref 0 00000100:00100000:0.0:1789760766.031303:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.031385:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515758976 00000100:00100000:3.0:1789760766.031396:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515758976 00000100:00100000:3.0:1789760766.031477:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744424253630 time 744461241196 state 0x0 ref 1 00000100:00100000:3.0:1789760766.031492:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744461501612 time 744461241196 state 0x1 ref 1 00000100:00100000:3.0:1789760766.031522:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744461501612 time 744496811406 state 0x0 ref 1 00000100:00100000:3.0:1789760766.031533:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515758976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760766.031578:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24272896, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760766.031585:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.031601:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760766.031606:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.031612:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.031625:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760766.032298:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760766.059002:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24272896 -> 24276992 00010000:00010000:3.0:1789760766.059023:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.059036:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.059043:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.059048:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.059052:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.059058:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.059075:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.059083:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f474 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.059177:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515758976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27637us (27873us total) trans 4294973243 rc 0/0 00000100:00100000:3.0:1789760766.059189:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5955 00000100:00100000:3.0:1789760766.059198:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744461501612 time 744496811406 state 0x2 ref 0 00000001:00080000:3.0:1789760766.059233:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973243 is committed 00000100:00100000:1.0:1789760766.063169:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.063248:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759104 00000100:00100000:3.0:1789760766.063259:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759104 00000100:00100000:3.0:1789760766.063308:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744461501612 time 744496811406 state 0x0 ref 1 00000100:00100000:3.0:1789760766.063323:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744497071822 time 744496811406 state 0x1 ref 1 00000100:00100000:3.0:1789760766.063355:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744497071822 time 744528644239 state 0x0 ref 1 00000100:00100000:3.0:1789760766.063367:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515759104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760766.063412:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24276992, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760766.063421:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.063442:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760766.063554:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.063561:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.063574:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.088582:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973244 is committed 00002000:00010000:3.0:1789760766.088819:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24276992 -> 24281088 00010000:00010000:3.0:1789760766.088836:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.088848:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.088857:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.088865:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.088869:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.088879:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.088898:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.088906:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f47b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.089003:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515759104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25633us (25837us total) trans 4294973244 rc 0/0 00000100:00100000:3.0:1789760766.089013:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5956 00000100:00100000:3.0:1789760766.089024:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744497071822 time 744528644239 state 0x2 ref 0 00000100:00100000:0.0:1789760766.094377:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.094458:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759232 00000100:00100000:3.0:1789760766.094467:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759232 00000100:00100000:3.0:1789760766.094517:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744497071822 time 744528644239 state 0x0 ref 1 00000100:00100000:3.0:1789760766.094530:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744528904655 time 744528644239 state 0x1 ref 1 00000100:00100000:3.0:1789760766.094565:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744528904655 time 744559847534 state 0x0 ref 1 00000100:00100000:3.0:1789760766.094577:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760766.094624:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24281088, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760766.094633:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.094650:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760766.094654:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.094661:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.094680:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.138422:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973245 is committed 00002000:00010000:3.0:1789760766.138532:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24281088 -> 24285184 00010000:00010000:3.0:1789760766.138546:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.138559:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.138569:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.138577:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.138582:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.138661:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.138681:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.138690:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f482 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.138818:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44237us (44444us total) trans 4294973245 rc 0/0 00000100:00100000:3.0:1789760766.138828:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5957 00000100:00100000:3.0:1789760766.139240:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744528904655 time 744559847534 state 0x2 ref 0 00000100:00100000:0.0:1789760766.143038:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.143400:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759360 00000100:00100000:3.0:1789760766.143410:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759360 00000100:00100000:3.0:1789760766.143472:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744528904655 time 744559847534 state 0x0 ref 1 00000100:00100000:3.0:1789760766.143488:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744560107950 time 744559847534 state 0x1 ref 1 00000100:00100000:3.0:1789760766.143518:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744560107950 time 744608807172 state 0x0 ref 1 00000100:00100000:3.0:1789760766.143531:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760766.143575:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24285184, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760766.143594:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.143612:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760766.143617:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.143623:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.143637:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.186090:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973246 is committed 00002000:00010000:3.0:1789760766.186195:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24285184 -> 24289280 00010000:00010000:3.0:1789760766.186212:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186225:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186234:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186242:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.186247:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186256:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186274:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.186282:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f489 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.186367:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42833us (43333us total) trans 4294973246 rc 0/0 00000100:00100000:3.0:1789760766.186380:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5958 00000100:00100000:3.0:1789760766.186389:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744560107950 time 744608807172 state 0x2 ref 0 00000100:00100000:0.0:1789760766.190322:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760766.190409:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759488 00000100:00100000:2.0:1789760766.190419:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759488 00000100:00100000:2.0:1789760766.190483:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744560107950 time 744608807172 state 0x0 ref 1 00000100:00100000:2.0:1789760766.190499:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744609067588 time 744608807172 state 0x1 ref 1 00000100:00100000:2.0:1789760766.190532:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744609067588 time 744655820829 state 0x0 ref 1 00000100:00100000:2.0:1789760766.190545:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.190813:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24289280, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.190824:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190840:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760766.190844:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.190850:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.190864:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.226292:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24289280 -> 24293376 00010000:00010000:2.0:1789760766.226311:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.226325:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.226335:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.226342:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760766.226348:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.226357:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.226375:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.226383:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f490 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.226484:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515759488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35936us (36166us total) trans 4294973247 rc 0/0 00000100:00100000:2.0:1789760766.226495:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5959 00000100:00100000:2.0:1789760766.226505:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744609067588 time 744655820829 state 0x2 ref 0 00000001:00080000:0.0:1789760766.229224:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973247 is committed 00000100:00100000:0.0:1789760766.230134:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760766.230209:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759616 00000100:00100000:2.0:1789760766.230220:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759616 00000100:00100000:2.0:1789760766.230272:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744609067588 time 744655820829 state 0x0 ref 1 00000100:00100000:2.0:1789760766.230287:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744656081245 time 744655820829 state 0x1 ref 1 00000100:00100000:2.0:1789760766.230315:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744656081245 time 744695604788 state 0x0 ref 1 00000100:00100000:2.0:1789760766.230327:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.230374:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24293376, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.230384:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.230400:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760766.230405:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.230411:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.230424:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.260066:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973248 is committed 00002000:00010000:1.0:1789760766.260196:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24293376 -> 24297472 00010000:00010000:1.0:1789760766.260212:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.260225:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.260235:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.260243:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760766.260247:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.260257:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.260277:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.260285:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f497 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.260382:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30050us (30248us total) trans 4294973248 rc 0/0 00000100:00100000:1.0:1789760766.260392:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5960 00000100:00100000:1.0:1789760766.260402:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744656081245 time 744695604788 state 0x2 ref 0 00000100:00100000:3.0:1789760766.263910:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760766.264015:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759744 00000100:00100000:0.0:1789760766.264026:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759744 00000100:00100000:0.0:1789760766.264077:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744656081245 time 744695604788 state 0x0 ref 1 00000100:00100000:0.0:1789760766.264104:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744695865204 time 744695604788 state 0x1 ref 1 00000100:00100000:0.0:1789760766.264170:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744695865204 time 744729458788 state 0x0 ref 1 00000100:00100000:0.0:1789760766.264183:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760766.264228:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24297472, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:0.0:1789760766.264237:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.264252:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760766.264256:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.264262:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.264274:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.273669:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760766.295746:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973249 is committed 00002000:00010000:3.0:1789760766.296205:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24297472 -> 24301568 00010000:00010000:3.0:1789760766.296221:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.296235:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.296245:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.296252:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.296257:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.296266:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.296286:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.296294:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1f49e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.296393:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515759744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32208us (32487us total) trans 4294973249 rc 0/0 00000100:00100000:3.0:1789760766.296402:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5961 00000100:00100000:3.0:1789760766.296411:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744695865204 time 744729458788 state 0x2 ref 0 00000100:00100000:2.0:1789760766.299380:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760766.300019:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515759872 00000100:00100000:0.0:1789760766.300030:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515759872 00000100:00100000:0.0:1789760766.300091:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744695865204 time 744729458788 state 0x0 ref 1 00000100:00100000:0.0:1789760766.300106:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744729719204 time 744729458788 state 0x1 ref 1 00000100:00100000:0.0:1789760766.300137:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744729719204 time 744765426219 state 0x0 ref 1 00000100:00100000:0.0:1789760766.300149:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515759872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760766.300197:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24301568, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:0.0:1789760766.300207:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.300223:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760766.300227:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.300233:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.300246:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.331594:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973250 is committed 00002000:00010000:3.0:1789760766.331693:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24301568 -> 24305664 00010000:00010000:3.0:1789760766.331737:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.331751:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.331761:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.331769:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.331773:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.331783:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.331816:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.331830:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f4a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.331922:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515759872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31770us (32548us total) trans 4294973250 rc 0/0 00000100:00100000:3.0:1789760766.331932:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5962 00000100:00100000:3.0:1789760766.331941:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744729719204 time 744765426219 state 0x2 ref 0 00000100:00100000:1.0:1789760766.338421:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.343425:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760000 00000100:00100000:3.0:1789760766.343439:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760000 00000100:00100000:2.0:1789760766.346481:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744729719204 time 744765426219 state 0x0 ref 1 00000100:00100000:2.0:1789760766.346500:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744765686635 time 744765426219 state 0x1 ref 1 00000100:00100000:2.0:1789760766.346585:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744765686635 time 744811873741 state 0x0 ref 1 00000100:00100000:2.0:1789760766.346599:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515760000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.346651:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24305664, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.346660:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.346676:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760766.346680:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.346688:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.346702:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.383920:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24305664 -> 24309760 00010000:00010000:2.0:1789760766.383950:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.383963:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.383972:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.383979:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760766.383985:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.383993:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.384012:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.384021:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f4ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.384115:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515760000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37514us (45697us total) trans 4294973251 rc 0/0 00000100:00100000:2.0:1789760766.384125:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5963 00000100:00100000:2.0:1789760766.384135:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744765686635 time 744811873741 state 0x2 ref 0 00000001:00080000:1.0:1789760766.384256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973251 is committed 00000100:00100000:2.0:1789760766.388087:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.389348:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760128 00000100:00100000:3.0:1789760766.389360:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760128 00000100:00100000:2.0:1789760766.392221:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744765686635 time 744811873741 state 0x0 ref 1 00000100:00100000:2.0:1789760766.392308:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744812134157 time 744811873741 state 0x1 ref 1 00000100:00100000:2.0:1789760766.392402:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744812134157 time 744857689691 state 0x0 ref 1 00000100:00100000:2.0:1789760766.392418:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515760128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.392471:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24309760, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.392480:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.392497:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760766.392501:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.392508:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.392522:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.432162:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973252 is committed 00002000:00010000:3.0:1789760766.432303:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24309760 -> 24313856 00010000:00010000:3.0:1789760766.432319:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.432332:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.432341:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.432348:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.432352:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.432365:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.432383:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.432390:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1f4b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.432523:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515760128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40103us (44440us total) trans 4294973252 rc 0/0 00000100:00100000:3.0:1789760766.432533:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5964 00000100:00100000:3.0:1789760766.432543:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744812134157 time 744857689691 state 0x2 ref 0 00000100:00100000:2.0:1789760766.437405:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760766.442393:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760256 00000100:00100000:1.0:1789760766.442410:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760256 00000100:00100000:1.0:1789760766.442473:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744812134157 time 744857689691 state 0x0 ref 1 00000100:00100000:1.0:1789760766.442487:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744857950107 time 744857689691 state 0x1 ref 1 00000100:00100000:1.0:1789760766.443913:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744857950107 time 744909194387 state 0x0 ref 1 00000100:00100000:1.0:1789760766.443936:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515760256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760766.443988:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24313856, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:1.0:1789760766.443997:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.444013:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760766.444018:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.444024:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.444038:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.472827:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973253 is committed 00002000:00010000:0.0:1789760766.472965:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24313856 -> 24317952 00010000:00010000:0.0:1789760766.472981:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.472994:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.473003:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.473010:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.473016:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.473025:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.473050:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.473058:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f4ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.473240:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515760256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29306us (35849us total) trans 4294973253 rc 0/0 00000100:00100000:0.0:1789760766.473251:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5965 00000100:00100000:0.0:1789760766.473260:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744857950107 time 744909194387 state 0x2 ref 0 00000100:00100000:1.0:1789760766.480052:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760766.480151:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760384 00000100:00100000:2.0:1789760766.480162:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760384 00000100:00100000:2.0:1789760766.480214:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744857950107 time 744909194387 state 0x0 ref 1 00000100:00100000:2.0:1789760766.480230:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744909454803 time 744909194387 state 0x1 ref 1 00000100:00100000:2.0:1789760766.480261:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744909454803 time 744945550003 state 0x0 ref 1 00000100:00100000:2.0:1789760766.480274:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515760384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.480325:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24317952, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.480334:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.480349:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760766.480353:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.480360:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.480373:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.508784:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973254 is committed 00002000:00010000:0.0:1789760766.508893:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24317952 -> 24322048 00010000:00010000:0.0:1789760766.508909:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.508923:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.508939:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.508947:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.508952:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.508969:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.508988:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.508995:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f4c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.509093:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515760384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28808us (29037us total) trans 4294973254 rc 0/0 00000100:00100000:0.0:1789760766.509104:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5966 00000100:00100000:0.0:1789760766.509113:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744909454803 time 744945550003 state 0x2 ref 0 00000100:00100000:2.0:1789760766.512503:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.512642:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760512 00000100:00100000:3.0:1789760766.512651:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760512 00000100:00100000:3.0:1789760766.512703:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744909454803 time 744945550003 state 0x0 ref 1 00000100:00100000:3.0:1789760766.512949:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744945810419 time 744945550003 state 0x1 ref 1 00000100:00100000:3.0:1789760766.513022:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744945810419 time 744978310826 state 0x0 ref 1 00000100:00100000:3.0:1789760766.513042:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515760512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760766.513090:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24322048, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760766.513099:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.513115:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760766.513119:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.513125:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.513149:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.542325:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973255 is committed 00002000:00010000:0.0:1789760766.542434:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24322048 -> 24326144 00010000:00010000:0.0:1789760766.542450:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.542464:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.542475:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.542482:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.542488:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.542496:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.542530:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.542538:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f4c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.542634:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515760512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29596us (30134us total) trans 4294973255 rc 0/0 00000100:00100000:0.0:1789760766.542644:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5967 00000100:00100000:0.0:1789760766.542654:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744945810419 time 744978310826 state 0x2 ref 0 00000100:00100000:1.0:1789760766.550213:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760766.550320:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760640 00000100:00100000:3.0:1789760766.550331:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760640 00000100:00100000:3.0:1789760766.550387:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744945810419 time 744978310826 state 0x0 ref 1 00000100:00100000:3.0:1789760766.550403:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744978571242 time 744978310826 state 0x1 ref 1 00000100:00100000:3.0:1789760766.550483:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744978571242 time 745015771222 state 0x0 ref 1 00000100:00100000:3.0:1789760766.550497:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515760640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760766.550560:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24326144, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:3.0:1789760766.550570:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.550597:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760766.550603:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.550610:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.550630:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.589035:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24326144 -> 24330240 00010000:00010000:0.0:1789760766.589053:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.589066:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.589076:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.589089:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.589094:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.589103:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.589122:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.589129:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f4cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.589219:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515760640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38719us (39011us total) trans 4294973256 rc 0/0 00000100:00100000:0.0:1789760766.589229:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5968 00000100:00100000:0.0:1789760766.589239:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744978571242 time 745015771222 state 0x2 ref 0 00000001:00080000:2.0:1789760766.589308:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973256 is committed 00000100:00100000:1.0:1789760766.592341:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760766.592862:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760768 00000100:00100000:0.0:1789760766.592873:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760768 00000100:00100000:0.0:1789760766.592923:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 744978571242 time 745015771222 state 0x0 ref 1 00000100:00100000:0.0:1789760766.592938:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745016031638 time 745015771222 state 0x1 ref 1 00000100:00100000:0.0:1789760766.593235:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745016031638 time 745058522345 state 0x0 ref 1 00000100:00100000:0.0:1789760766.593251:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515760768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760766.593299:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24330240, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:0.0:1789760766.593309:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.593329:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760766.593333:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.593339:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.593401:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.650421:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24330240 -> 24334336 00010000:00010000:0.0:1789760766.650442:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.650456:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.650465:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.650473:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.650477:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.650488:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.650506:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.650513:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.650614:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515760768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57353us (58271us total) trans 4294973257 rc 0/0 00000100:00100000:0.0:1789760766.650625:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5969 00000100:00100000:0.0:1789760766.650643:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745016031638 time 745058522345 state 0x2 ref 0 00000001:00080000:3.0:1789760766.652153:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973257 is committed 00000100:00100000:3.0:1789760766.654695:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760766.654909:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515760896 00000100:00100000:2.0:1789760766.654920:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515760896 00000100:00100000:2.0:1789760766.654987:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745016031638 time 745058522345 state 0x0 ref 1 00000100:00100000:2.0:1789760766.655002:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745058782761 time 745058522345 state 0x1 ref 1 00000100:00100000:2.0:1789760766.655033:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745058782761 time 745120322244 state 0x0 ref 1 00000100:00100000:2.0:1789760766.655045:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515760896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.655107:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24334336, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.655117:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.655133:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.655138:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.655144:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.655159:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.709074:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973258 is committed 00002000:00010000:0.0:1789760766.709253:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24334336 -> 24338432 00010000:00010000:0.0:1789760766.709269:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.709283:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.709293:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.709300:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.709306:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.709316:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.709335:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.709342:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f4dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.709428:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515760896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54381us (54738us total) trans 4294973258 rc 0/0 00000100:00100000:0.0:1789760766.709438:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5970 00000100:00100000:0.0:1789760766.709447:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745058782761 time 745120322244 state 0x2 ref 0 00000100:00100000:3.0:1789760766.716025:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760766.716299:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761024 00000100:00100000:1.0:1789760766.716311:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761024 00000100:00100000:1.0:1789760766.716370:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745058782761 time 745120322244 state 0x0 ref 1 00000100:00100000:1.0:1789760766.716387:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745120582660 time 745120322244 state 0x1 ref 1 00000100:00100000:1.0:1789760766.716419:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745120582660 time 745181707915 state 0x0 ref 1 00000100:00100000:1.0:1789760766.716430:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515761024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760766.716486:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24338432, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:1.0:1789760766.716496:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.716512:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.716517:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.716523:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.716537:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.754077:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973259 is committed 00002000:00010000:0.0:1789760766.754288:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24338432 -> 24342528 00010000:00010000:0.0:1789760766.754303:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.754317:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.754326:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.754339:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.754344:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.754352:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.754372:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.754380:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f4e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.754473:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515761024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38040us (38453us total) trans 4294973259 rc 0/0 00000100:00100000:0.0:1789760766.754485:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5971 00000100:00100000:0.0:1789760766.754494:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745120582660 time 745181707915 state 0x2 ref 0 00000100:00100000:1.0:1789760766.760405:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760766.760498:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761152 00000100:00100000:2.0:1789760766.760507:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761152 00000100:00100000:2.0:1789760766.760557:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745120582660 time 745181707915 state 0x0 ref 1 00000100:00100000:2.0:1789760766.760572:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745181968331 time 745181707915 state 0x1 ref 1 00000100:00100000:2.0:1789760766.760604:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745181968331 time 745225892882 state 0x0 ref 1 00000100:00100000:2.0:1789760766.760619:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515761152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.760667:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24342528, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:2.0:1789760766.760676:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.760692:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.760696:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.760702:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.760881:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.814669:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973260 is committed 00002000:00010000:0.0:1789760766.814920:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24342528 -> 24346624 00010000:00010000:0.0:1789760766.814936:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.814949:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.814965:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.814972:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.814978:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.814986:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.815006:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.815014:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f4eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.815200:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515761152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54577us (54798us total) trans 4294973260 rc 0/0 00000100:00100000:0.0:1789760766.815209:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5972 00000100:00100000:0.0:1789760766.815218:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745181968331 time 745225892882 state 0x2 ref 0 00000100:00100000:3.0:1789760766.818902:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760766.819054:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761280 00000100:00100000:0.0:1789760766.819064:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761280 00000100:00100000:0.0:1789760766.819127:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745181968331 time 745225892882 state 0x0 ref 1 00000100:00100000:0.0:1789760766.819142:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745226153298 time 745225892882 state 0x1 ref 1 00000100:00100000:0.0:1789760766.819164:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745226153298 time 745284454464 state 0x0 ref 1 00000100:00100000:0.0:1789760766.819175:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515761280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760766.819220:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24346624, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc007 00010000:00010000:0.0:1789760766.819229:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.819243:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760766.819247:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.819252:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.819266:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760766.850397:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24346624 -> 24350720 00002000:00010000:0.0:1789760766.850406:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760766 -> 1789760767 00002000:00010000:0.0:1789760766.850409:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760766 -> 1789760767 00010000:00010000:0.0:1789760766.850422:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.850506:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.850517:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.850526:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.850531:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.850540:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.850560:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.850568:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f4f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.850660:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515761280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31482us (31765us total) trans 4294973261 rc 0/0 00000100:00100000:0.0:1789760766.850671:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5973 00000100:00100000:0.0:1789760766.850680:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745226153298 time 745284454464 state 0x2 ref 0 00000100:00100000:1.0:1789760766.854417:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:3.0:1789760766.854753:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973261 is committed 00000100:00100000:0.0:1789760766.854883:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761408 00000100:00100000:0.0:1789760766.854893:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761408 00000100:00100000:0.0:1789760766.854945:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745226153298 time 745284454464 state 0x0 ref 1 00000100:00100000:0.0:1789760766.854960:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745284714880 time 745284454464 state 0x1 ref 1 00000100:00100000:0.0:1789760766.854982:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745284714880 time 745320271404 state 0x0 ref 1 00000100:00100000:0.0:1789760766.854999:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515761408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760766.855046:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24350720, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760766.855055:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.855070:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760766.855074:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.855080:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.855093:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.900601:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973262 is committed 00002000:00010000:3.0:1789760766.900685:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24350720 -> 24354816 00010000:00010000:3.0:1789760766.900702:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.900811:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.900820:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.900827:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760766.900832:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.900842:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.900865:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.900873:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f4f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.900957:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515761408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45956us (46544us total) trans 4294973262 rc 0/0 00000100:00100000:3.0:1789760766.900968:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5974 00000100:00100000:3.0:1789760766.900978:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745284714880 time 745320271404 state 0x2 ref 0 00000100:00100000:1.0:1789760766.904097:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760766.904200:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761536 00000100:00100000:0.0:1789760766.904210:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761536 00000100:00100000:0.0:1789760766.904261:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745284714880 time 745320271404 state 0x0 ref 1 00000100:00100000:0.0:1789760766.904277:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745320531820 time 745320271404 state 0x1 ref 1 00000100:00100000:0.0:1789760766.904371:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745320531820 time 745369658816 state 0x0 ref 1 00000100:00100000:0.0:1789760766.904384:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515761536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760766.904431:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24354816, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760766.904441:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.904461:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760766.904465:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.904471:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.904484:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.931514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973263 is committed 00002000:00010000:0.0:1789760766.931622:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24354816 -> 24358912 00010000:00010000:0.0:1789760766.931639:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.931653:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.931663:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.931671:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760766.931676:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.931686:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.931901:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.931912:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f500 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.931997:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515761536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27610us (27903us total) trans 4294973263 rc 0/0 00000100:00100000:0.0:1789760766.932008:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5975 00000100:00100000:0.0:1789760766.932018:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745320531820 time 745369658816 state 0x2 ref 0 00000100:00100000:2.0:1789760766.942282:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760766.944894:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761664 00000100:00100000:1.0:1789760766.944908:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761664 00000100:00100000:2.0:1789760766.948265:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745320531820 time 745369658816 state 0x0 ref 1 00000100:00100000:2.0:1789760766.948285:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745369919232 time 745369658816 state 0x1 ref 1 00000100:00100000:2.0:1789760766.948318:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745369919232 time 745413607619 state 0x0 ref 1 00000100:00100000:2.0:1789760766.948451:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515761664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760766.948502:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24358912, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:2.0:1789760766.948512:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.948533:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.948538:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.948545:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.948572:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760767.007726:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24358912 -> 24363008 00010000:00010000:2.0:1789760767.007747:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.007761:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.007770:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.007778:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.007783:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.007801:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.007821:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.007830:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f507 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.007916:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515761664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59467us (65639us total) trans 4294973264 rc 0/0 00000100:00100000:2.0:1789760767.007926:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5976 00000100:00100000:2.0:1789760767.007935:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745369919232 time 745413607619 state 0x2 ref 0 00000001:00080000:3.0:1789760767.008458:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973264 is committed 00000100:00100000:1.0:1789760767.016260:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.016354:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761792 00000100:00100000:0.0:1789760767.016364:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761792 00000100:00100000:0.0:1789760767.016417:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745369919232 time 745413607619 state 0x0 ref 1 00000100:00100000:0.0:1789760767.016432:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745413868035 time 745413607619 state 0x1 ref 1 00000100:00100000:0.0:1789760767.016459:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745413868035 time 745481749088 state 0x0 ref 1 00000100:00100000:0.0:1789760767.016472:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515761792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.016521:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24363008, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.016529:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.016545:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760767.016549:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.016555:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.016572:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.049389:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24363008 -> 24367104 00010000:00010000:0.0:1789760767.049407:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.049422:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.049432:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.049511:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760767.049517:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.049527:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.049546:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.049555:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f50e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.049680:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515761792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33205us (33424us total) trans 4294973265 rc 0/0 00000100:00100000:0.0:1789760767.049747:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5977 00000100:00100000:0.0:1789760767.049758:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745413868035 time 745481749088 state 0x2 ref 0 00000001:00080000:3.0:1789760767.050073:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973265 is committed 00000100:00100000:0.0:1789760767.054809:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.054867:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515761920 00000100:00100000:3.0:1789760767.054877:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515761920 00000100:00100000:3.0:1789760767.054927:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745413868035 time 745481749088 state 0x0 ref 1 00000100:00100000:3.0:1789760767.054942:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745482009504 time 745481749088 state 0x1 ref 1 00000100:00100000:3.0:1789760767.054971:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745482009504 time 745520260184 state 0x0 ref 1 00000100:00100000:3.0:1789760767.054984:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515761920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760767.055032:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24367104, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:3.0:1789760767.055042:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.055057:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760767.055061:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.055068:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.055081:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.098242:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24367104 -> 24371200 00010000:00010000:0.0:1789760767.098264:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.098277:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.098285:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.098292:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760767.098296:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.098305:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.098324:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.098337:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f515 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.098432:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515761920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43445us (43637us total) trans 4294973266 rc 0/0 00000100:00100000:0.0:1789760767.098442:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5978 00000100:00100000:0.0:1789760767.098452:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745482009504 time 745520260184 state 0x2 ref 0 00000001:00080000:3.0:1789760767.098883:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973266 is committed 00000100:00100000:2.0:1789760767.102939:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.103014:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762048 00000100:00100000:0.0:1789760767.103023:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762048 00000100:00100000:0.0:1789760767.103074:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745482009504 time 745520260184 state 0x0 ref 1 00000100:00100000:0.0:1789760767.103090:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745520520600 time 745520260184 state 0x1 ref 1 00000100:00100000:0.0:1789760767.103119:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745520520600 time 745568408691 state 0x0 ref 1 00000100:00100000:0.0:1789760767.103131:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.103179:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24371200, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.103188:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.103204:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760767.103208:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.103214:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.103233:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.127404:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973267 is committed 00002000:00010000:0.0:1789760767.127490:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24371200 -> 24375296 00010000:00010000:0.0:1789760767.127505:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.127518:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.127528:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.127536:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760767.127540:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.127548:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.127568:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.127575:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f51c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.127746:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24573us (24772us total) trans 4294973267 rc 0/0 00000100:00100000:0.0:1789760767.127755:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5979 00000100:00100000:0.0:1789760767.127765:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745520520600 time 745568408691 state 0x2 ref 0 00000100:00100000:1.0:1789760767.131036:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.131328:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762176 00000100:00100000:3.0:1789760767.131338:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762176 00000100:00100000:3.0:1789760767.131391:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745520520600 time 745568408691 state 0x0 ref 1 00000100:00100000:3.0:1789760767.131406:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745568669107 time 745568408691 state 0x1 ref 1 00000100:00100000:3.0:1789760767.131478:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745568669107 time 745596766624 state 0x0 ref 1 00000100:00100000:3.0:1789760767.131492:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515762176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760767.131540:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24375296, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:3.0:1789760767.131549:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.131565:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760767.131569:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.131576:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.131589:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.156540:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973268 is committed 00002000:00010000:2.0:1789760767.156645:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24375296 -> 24379392 00010000:00010000:2.0:1789760767.156661:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.156675:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.156683:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.156690:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.156694:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.156702:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.156875:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.156883:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f523 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.157015:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515762176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25520us (25983us total) trans 4294973268 rc 0/0 00000100:00100000:2.0:1789760767.157025:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5980 00000100:00100000:2.0:1789760767.157035:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745568669107 time 745596766624 state 0x2 ref 0 00000100:00100000:1.0:1789760767.160162:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.160247:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762304 00000100:00100000:0.0:1789760767.160256:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762304 00000100:00100000:0.0:1789760767.160303:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745568669107 time 745596766624 state 0x0 ref 1 00000100:00100000:0.0:1789760767.160316:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745597027040 time 745596766624 state 0x1 ref 1 00000100:00100000:0.0:1789760767.160344:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745597027040 time 745625634206 state 0x0 ref 1 00000100:00100000:0.0:1789760767.160355:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.160402:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24379392, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.160411:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.160426:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760767.160430:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.160529:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.160545:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.190526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973269 is committed 00002000:00010000:2.0:1789760767.190627:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24379392 -> 24383488 00010000:00010000:2.0:1789760767.190642:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.190654:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.190664:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.190671:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.190676:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.190690:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.190803:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.190812:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f52a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.190895:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30537us (30737us total) trans 4294973269 rc 0/0 00000100:00100000:2.0:1789760767.190906:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5981 00000100:00100000:2.0:1789760767.190915:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745597027040 time 745625634206 state 0x2 ref 0 00000100:00100000:2.0:1789760767.195278:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.195369:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762432 00000100:00100000:0.0:1789760767.195380:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762432 00000100:00100000:0.0:1789760767.195499:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745597027040 time 745625634206 state 0x0 ref 1 00000100:00100000:0.0:1789760767.195516:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745625894622 time 745625634206 state 0x1 ref 1 00000100:00100000:0.0:1789760767.195582:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745625894622 time 745660870777 state 0x0 ref 1 00000100:00100000:0.0:1789760767.195596:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.195644:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24383488, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.195653:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.195667:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760767.195671:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.195676:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.195689:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.249413:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973270 is committed 00002000:00010000:2.0:1789760767.249646:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24383488 -> 24387584 00010000:00010000:2.0:1789760767.249662:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.249675:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.249685:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.249698:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.249704:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.249743:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.249763:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.249771:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f531 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.249875:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54276us (54602us total) trans 4294973270 rc 0/0 00000100:00100000:2.0:1789760767.249886:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5982 00000100:00100000:2.0:1789760767.249896:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745625894622 time 745660870777 state 0x2 ref 0 00000100:00100000:3.0:1789760767.253056:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.253137:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762560 00000100:00100000:0.0:1789760767.253147:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762560 00000100:00100000:0.0:1789760767.253209:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745625894622 time 745660870777 state 0x0 ref 1 00000100:00100000:0.0:1789760767.253223:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745661131193 time 745660870777 state 0x1 ref 1 00000100:00100000:0.0:1789760767.253251:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745661131193 time 745718539817 state 0x0 ref 1 00000100:00100000:0.0:1789760767.253263:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.253310:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24387584, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.253320:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.253336:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760767.253341:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.253347:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.253361:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.284474:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973271 is committed 00002000:00010000:0.0:1789760767.284637:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24387584 -> 24391680 00010000:00010000:0.0:1789760767.284652:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.284664:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.284674:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.284681:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760767.284687:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.284695:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.284854:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.284864:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f538 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.284992:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31727us (31941us total) trans 4294973271 rc 0/0 00000100:00100000:0.0:1789760767.285003:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5983 00000100:00100000:0.0:1789760767.285012:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745661131193 time 745718539817 state 0x2 ref 0 00000100:00100000:0.0:1789760767.289019:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.289114:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762688 00000100:00100000:3.0:1789760767.289124:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762688 00000100:00100000:3.0:1789760767.289179:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745661131193 time 745718539817 state 0x0 ref 1 00000100:00100000:3.0:1789760767.289193:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745718800233 time 745718539817 state 0x1 ref 1 00000100:00100000:3.0:1789760767.289264:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745718800233 time 745754552774 state 0x0 ref 1 00000100:00100000:3.0:1789760767.289276:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515762688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760767.289323:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24391680, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:3.0:1789760767.289332:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.289347:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760767.289351:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.289358:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.289370:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.297942:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760767.337664:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973272 is committed 00002000:00010000:3.0:1789760767.338118:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24391680 -> 24395776 00010000:00010000:3.0:1789760767.338134:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.338148:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.338158:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.338165:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760767.338171:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.338180:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.338199:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.338208:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f53f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.338300:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515762688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49020us (49286us total) trans 4294973272 rc 0/0 00000100:00100000:3.0:1789760767.338314:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5984 00000100:00100000:3.0:1789760767.338323:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745718800233 time 745754552774 state 0x2 ref 0 00000100:00100000:3.0:1789760767.341309:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.343022:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762816 00000100:00100000:0.0:1789760767.343037:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762816 00000100:00100000:0.0:1789760767.343095:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745718800233 time 745754552774 state 0x0 ref 1 00000100:00100000:0.0:1789760767.343110:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745754813190 time 745754552774 state 0x1 ref 1 00000100:00100000:0.0:1789760767.343147:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745754813190 time 745808436184 state 0x0 ref 1 00000100:00100000:0.0:1789760767.343161:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515762816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.343214:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24395776, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.343223:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.343237:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760767.343241:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.343248:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.343263:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.390221:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24395776 -> 24399872 00010000:00010000:0.0:1789760767.390242:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.390255:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.390265:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.390273:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760767.390279:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.390290:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.390311:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.390319:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f546 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.390418:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515762816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47255us (49114us total) trans 4294973273 rc 0/0 00000100:00100000:0.0:1789760767.390428:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5985 00000100:00100000:0.0:1789760767.390446:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745754813190 time 745808436184 state 0x2 ref 0 00000001:00080000:3.0:1789760767.392556:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973273 is committed 00000100:00100000:1.0:1789760767.394966:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.395037:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515762944 00000100:00100000:2.0:1789760767.395046:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515762944 00000100:00100000:2.0:1789760767.395100:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745754813190 time 745808436184 state 0x0 ref 1 00000100:00100000:2.0:1789760767.395116:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745808696600 time 745808436184 state 0x1 ref 1 00000100:00100000:2.0:1789760767.395143:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745808696600 time 745860431795 state 0x0 ref 1 00000100:00100000:2.0:1789760767.395156:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.395207:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24399872, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:2.0:1789760767.395216:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.395232:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760767.395237:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.395243:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.395257:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.440294:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973274 is committed 00002000:00010000:1.0:1789760767.440543:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24399872 -> 24403968 00010000:00010000:1.0:1789760767.440558:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.440569:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.440578:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.440585:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760767.440590:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.440600:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.440618:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.440626:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f54d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.440863:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515762944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45704us (45900us total) trans 4294973274 rc 0/0 00000100:00100000:1.0:1789760767.440874:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5986 00000100:00100000:1.0:1789760767.440884:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745808696600 time 745860431795 state 0x2 ref 0 00000100:00100000:3.0:1789760767.444666:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.444774:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763072 00000100:00100000:2.0:1789760767.444784:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763072 00000100:00100000:2.0:1789760767.444845:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745808696600 time 745860431795 state 0x0 ref 1 00000100:00100000:2.0:1789760767.444860:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745860692211 time 745860431795 state 0x1 ref 1 00000100:00100000:2.0:1789760767.444884:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745860692211 time 745910173899 state 0x0 ref 1 00000100:00100000:2.0:1789760767.444906:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.444952:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24403968, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:2.0:1789760767.444962:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.444977:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760767.444981:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.444988:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.445000:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.485945:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c3bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544417024:0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.485973:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c3bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544417152:0@lo:400:kworker.0 00000100:00100000:3.0:1789760767.486020:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760767.486021:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804df0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544417408:0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.486028:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760767.486058:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760767.486074:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804df2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544417536:0@lo:400:kworker.0 00000100:00100000:2.0:1789760767.486088:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c3bdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544417280:0@lo:400:kworker.0 00000100:00100000:2.0:1789760767.486129:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417408 00000100:00100000:2.0:1789760767.486135:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417408 00000100:00080000:3.0:1789760767.486178:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760767.486187:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789760767.486196:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544417408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760767.486655:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544417408:12345-0@lo:400:kworker.0 Request processed in 458us (594us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760767.486663:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:0.0:1789760767.486800:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804df0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544417408:0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.494575:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544417664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760767.494575:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760767.494641:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760767.494693:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417024 00000100:00100000:1.0:1789760767.494701:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417024 00000100:00100000:1.0:1789760767.494767:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1789760767.494776:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544417024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760767.494824:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417152 00000100:00100000:0.0:1789760767.494829:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417152 00000100:00100000:0.0:1789760767.494858:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417280 00000100:00100000:0.0:1789760767.494862:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417280 00000100:00100000:0.0:1789760767.494888:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00080000:3.0:1789760767.494910:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760767.494947:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417664 00000100:00100000:2.0:1789760767.494953:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417664 00000100:00100000:0.0:1789760767.494963:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544417152:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760767.494997:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:2.0:1789760767.495006:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544417664:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789760767.495018:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760767.495256:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544417024:12345-0@lo:400:kworker.0 Request processed in 480us (9236us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760767.495264:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:1.0:1789760767.495311:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:1.0:1789760767.495320:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544417280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760767.499614:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544417664:12345-0@lo:400:kworker.0 Request processed in 4601us (4965us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760767.499629:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:2.0:1789760767.500339:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544417664:0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.500642:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544417280:12345-0@lo:400:kworker.0 Request processed in 5321us (6074us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760767.500652:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:1.0:1789760767.500755:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760767.500804:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c3bc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544417024:0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.500830:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417536 00000100:00100000:1.0:1789760767.500836:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417536 00000100:00100000:3.0:1789760767.500976:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c3bdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544417280:0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.501155:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:1.0:1789760767.501164:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544417536:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760767.501228:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544417536:12345-0@lo:400:kworker.0 Request processed in 65us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760767.501235:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00080000:1.0:1789760767.501254:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760767.501322:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804df2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544417536:0@lo:400:kworker.0 00000100:00100000:3.0:1789760767.503577:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544417152:12345-0@lo:400:kworker.0 Request processed in 8610us (17547us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760767.503590:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1789760767.503825:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c3bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544417152:0@lo:400:kworker.0 00000001:00080000:3.0:1789760767.503982:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973275 is committed 00002000:00010000:2.0:1789760767.504039:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24403968 -> 24408064 00010000:00010000:2.0:1789760767.504053:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.504066:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.504075:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.504083:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.504087:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.504096:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.504115:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.504123:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f554 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.504197:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59301us (59538us total) trans 4294973275 rc 0/0 00000100:00100000:2.0:1789760767.504209:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5987 00000100:00100000:2.0:1789760767.504226:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745860692211 time 745910173899 state 0x2 ref 0 00000100:00100000:1.0:1789760767.507055:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.507099:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763200 00000100:00100000:2.0:1789760767.507108:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763200 00000100:00100000:2.0:1789760767.507164:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745860692211 time 745910173899 state 0x0 ref 1 00000100:00100000:2.0:1789760767.507178:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745910434315 time 745910173899 state 0x1 ref 1 00000100:00100000:2.0:1789760767.507208:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745910434315 time 745972497524 state 0x0 ref 1 00000100:00100000:2.0:1789760767.507221:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.507268:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24408064, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:2.0:1789760767.507277:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.507293:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760767.507297:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.507304:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.507317:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.553538:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973276 is committed 00002000:00010000:3.0:1789760767.553748:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24408064 -> 24412160 00010000:00010000:3.0:1789760767.553766:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.553779:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.553796:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.553804:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760767.553808:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.553816:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.553836:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.553843:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1f55b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.553935:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46711us (46885us total) trans 4294973276 rc 0/0 00000100:00100000:3.0:1789760767.553944:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5988 00000100:00100000:3.0:1789760767.553954:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745910434315 time 745972497524 state 0x2 ref 0 00000100:00100000:1.0:1789760767.561320:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.561435:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763328 00000100:00100000:0.0:1789760767.561445:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763328 00000100:00100000:0.0:1789760767.561491:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745910434315 time 745972497524 state 0x0 ref 1 00000100:00100000:0.0:1789760767.561513:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745972757940 time 745972497524 state 0x1 ref 1 00000100:00100000:0.0:1789760767.561543:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745972757940 time 746026832673 state 0x0 ref 1 00000100:00100000:0.0:1789760767.561562:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.561639:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24412160, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:0.0:1789760767.561648:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.561667:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.561672:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.561678:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.561697:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.601530:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973277 is committed 00002000:00010000:2.0:1789760767.601733:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24412160 -> 24416256 00010000:00010000:2.0:1789760767.601749:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601761:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601770:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601777:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.601781:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601799:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601818:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.601826:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f562 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.601918:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40357us (40604us total) trans 4294973277 rc 0/0 00000100:00100000:2.0:1789760767.601928:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5989 00000100:00100000:2.0:1789760767.601937:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745972757940 time 746026832673 state 0x2 ref 0 00000100:00100000:2.0:1789760767.611146:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.611280:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763456 00000100:00100000:3.0:1789760767.611290:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763456 00000100:00100000:3.0:1789760767.611345:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 745972757940 time 746026832673 state 0x0 ref 1 00000100:00100000:3.0:1789760767.611360:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746027093089 time 746026832673 state 0x1 ref 1 00000100:00100000:3.0:1789760767.611388:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746027093089 time 746076677375 state 0x0 ref 1 00000100:00100000:3.0:1789760767.611401:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760767.611452:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24416256, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:3.0:1789760767.611461:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.611477:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.611482:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.611488:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.611501:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.653180:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973278 is committed 00002000:00010000:3.0:1789760767.653287:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24416256 -> 24420352 00010000:00010000:3.0:1789760767.653304:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.653317:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.653332:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.653338:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760767.653354:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.653364:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.653387:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.653394:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f569 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.653491:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42086us (42349us total) trans 4294973278 rc 0/0 00000100:00100000:3.0:1789760767.653500:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5990 00000100:00100000:3.0:1789760767.653510:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746027093089 time 746076677375 state 0x2 ref 0 00000100:00100000:0.0:1789760767.658089:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.658176:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763584 00000100:00100000:3.0:1789760767.658184:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763584 00000100:00100000:3.0:1789760767.658233:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746027093089 time 746076677375 state 0x0 ref 1 00000100:00100000:3.0:1789760767.658248:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746076937791 time 746076677375 state 0x1 ref 1 00000100:00100000:3.0:1789760767.658271:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746076937791 time 746123560886 state 0x0 ref 1 00000100:00100000:3.0:1789760767.658281:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760767.658326:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24420352, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:3.0:1789760767.658335:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.658360:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.658364:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.658370:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.658382:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760767.705498:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24420352 -> 24424448 00010000:00010000:2.0:1789760767.705518:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.705530:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.705538:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.705545:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.705550:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.705560:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.705579:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.705585:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f570 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.705680:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515763584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47394us (47593us total) trans 4294973279 rc 0/0 00000100:00100000:2.0:1789760767.705695:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5991 00000100:00100000:2.0:1789760767.705704:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746076937791 time 746123560886 state 0x2 ref 0 00000001:00080000:1.0:1789760767.706615:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973279 is committed 00000100:00100000:1.0:1789760767.713523:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.713586:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763712 00000100:00100000:2.0:1789760767.713596:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763712 00000100:00100000:2.0:1789760767.713647:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746076937791 time 746123560886 state 0x0 ref 1 00000100:00100000:2.0:1789760767.713661:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746123821302 time 746123560886 state 0x1 ref 1 00000100:00100000:2.0:1789760767.713835:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746123821302 time 746179114325 state 0x0 ref 1 00000100:00100000:2.0:1789760767.713848:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.713896:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24424448, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:2.0:1789760767.713905:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.713920:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.713924:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.713929:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.713942:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.774467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973280 is committed 00002000:00010000:1.0:1789760767.774567:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24424448 -> 24428544 00010000:00010000:1.0:1789760767.774581:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.774594:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.774603:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.774611:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760767.774615:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.774623:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.774646:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.774654:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f577 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.776554:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62694us (63031us total) trans 4294973280 rc 0/0 00000100:00100000:1.0:1789760767.776569:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5992 00000100:00100000:1.0:1789760767.776578:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746123821302 time 746179114325 state 0x2 ref 0 00000100:00100000:3.0:1789760767.780979:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.781175:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763840 00000100:00100000:2.0:1789760767.781186:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763840 00000100:00100000:2.0:1789760767.781235:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746123821302 time 746179114325 state 0x0 ref 1 00000100:00100000:2.0:1789760767.781249:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746179374741 time 746179114325 state 0x1 ref 1 00000100:00100000:2.0:1789760767.781281:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746179374741 time 746246569959 state 0x0 ref 1 00000100:00100000:2.0:1789760767.781292:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515763840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.781338:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24428544, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:2.0:1789760767.781347:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.781363:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.781367:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.781373:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.781389:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.831152:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24428544 -> 24432640 00010000:00010000:3.0:1789760767.831173:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831186:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831194:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831207:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760767.831211:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831220:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831240:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.831248:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f57e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.831331:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515763840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50036us (50357us total) trans 4294973281 rc 0/0 00000100:00100000:3.0:1789760767.831341:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5993 00000100:00100000:3.0:1789760767.831350:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746179374741 time 746246569959 state 0x2 ref 0 00000001:00080000:1.0:1789760767.831423:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973281 is committed 00000100:00100000:2.0:1789760767.835799:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.836116:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515763968 00000100:00100000:3.0:1789760767.836126:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515763968 00000100:00100000:3.0:1789760767.836186:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746179374741 time 746246569959 state 0x0 ref 1 00000100:00100000:3.0:1789760767.836202:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746246830375 time 746246569959 state 0x1 ref 1 00000100:00100000:3.0:1789760767.836269:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746246830375 time 746301557134 state 0x0 ref 1 00000100:00100000:3.0:1789760767.836285:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760767.836334:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24432640, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc007 00010000:00010000:3.0:1789760767.836344:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.836360:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.836364:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.836369:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.836382:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.881614:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973282 is committed 00002000:00010000:2.0:1789760767.881743:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24432640 -> 24436736 00002000:00010000:2.0:1789760767.881748:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760767 -> 1789760768 00002000:00010000:2.0:1789760767.881751:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760767 -> 1789760768 00010000:00010000:2.0:1789760767.881764:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.881782:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.881800:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.881808:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.881813:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.881822:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.881851:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.881859:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f585 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.881952:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515763968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45664us (46240us total) trans 4294973282 rc 0/0 00000100:00100000:2.0:1789760767.881962:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5994 00000100:00100000:2.0:1789760767.881972:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746246830375 time 746301557134 state 0x2 ref 0 00000100:00100000:2.0:1789760767.886208:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760767.886290:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764096 00000100:00100000:0.0:1789760767.886300:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764096 00000100:00100000:0.0:1789760767.886353:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746246830375 time 746301557134 state 0x0 ref 1 00000100:00100000:0.0:1789760767.886368:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746301817550 time 746301557134 state 0x1 ref 1 00000100:00100000:0.0:1789760767.886398:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746301817550 time 746351688220 state 0x0 ref 1 00000100:00100000:0.0:1789760767.886410:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760767.886464:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24436736, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760767.886473:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.886487:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.886491:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.886498:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.886511:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.925207:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.925278:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764224 00000100:00100000:3.0:1789760767.925289:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764224 00000100:00100000:2.0:1789760767.925343:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760767.925360:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 402 00000100:00100000:3.0:1789760767.925373:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515764224:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760767.925384:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.925411:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764480 00000100:00100000:2.0:1789760767.925416:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764480 00000100:00100000:3.0:1789760767.925450:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515764224:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 80us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760767.925458:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 402 00000100:00100000:2.0:1789760767.925541:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764352 00000100:00100000:2.0:1789760767.925546:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764352 00000100:00100000:2.0:1789760767.925570:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 481 00000100:00100000:2.0:1789760767.925579:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515764480:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760767.926140:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515764480:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 560us (794us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760767.926149:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 481 00000100:00100000:2.0:1789760767.926166:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 482 00000100:00100000:2.0:1789760767.926173:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515764352:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760767.926208:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515764352:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 36us (826us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760767.926214:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 482 00000100:00080000:2.0:1789760767.926234:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:2.0:1789760767.933380:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24436736 -> 24440832 00010000:00010000:2.0:1789760767.933401:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.933414:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.933423:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.933436:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.933509:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.933520:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.933541:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.933549:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f58c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.933654:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515764096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47240us (47450us total) trans 4294973283 rc 0/0 00000100:00100000:2.0:1789760767.933664:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5995 00000100:00100000:2.0:1789760767.934244:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746301817550 time 746351688220 state 0x2 ref 0 00000001:00080000:3.0:1789760767.936253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973283 is committed 00000100:00100000:3.0:1789760767.939340:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.939776:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764608 00000100:00100000:2.0:1789760767.939797:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764608 00000100:00100000:2.0:1789760767.939862:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746301817550 time 746351688220 state 0x0 ref 1 00000100:00100000:2.0:1789760767.939877:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746351948636 time 746351688220 state 0x1 ref 1 00000100:00100000:2.0:1789760767.939902:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746351948636 time 746405191685 state 0x0 ref 1 00000100:00100000:2.0:1789760767.939917:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.939962:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24440832, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760767.939971:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.939986:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.939990:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.939996:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.940009:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.972918:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973284 is committed 00002000:00010000:2.0:1789760767.973114:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24440832 -> 24444928 00010000:00010000:2.0:1789760767.973130:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.973143:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.973152:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.973159:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760767.973163:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.973172:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.973191:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.973199:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f593 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.973325:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33404us (33993us total) trans 4294973284 rc 0/0 00000100:00100000:2.0:1789760767.973336:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5996 00000100:00100000:2.0:1789760767.973346:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746351948636 time 746405191685 state 0x2 ref 0 00000100:00100000:1.0:1789760767.984819:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760767.984920:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764736 00000100:00100000:2.0:1789760767.984931:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764736 00000100:00100000:2.0:1789760767.984986:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746351948636 time 746405191685 state 0x0 ref 1 00000100:00100000:2.0:1789760767.985002:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746405452101 time 746405191685 state 0x1 ref 1 00000100:00100000:2.0:1789760767.985034:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746405452101 time 746450323338 state 0x0 ref 1 00000100:00100000:2.0:1789760767.985049:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760767.985099:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24444928, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760767.985108:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.985125:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.985130:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.985137:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.985152:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.025665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973285 is committed 00002000:00010000:3.0:1789760768.025903:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24444928 -> 24449024 00010000:00010000:3.0:1789760768.025920:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.025934:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.025943:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.025951:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760768.025956:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.025964:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.025984:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.025991:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f59a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.026084:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41034us (41271us total) trans 4294973285 rc 0/0 00000100:00100000:3.0:1789760768.026093:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5997 00000100:00100000:3.0:1789760768.026103:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746405452101 time 746450323338 state 0x2 ref 0 00000100:00100000:3.0:1789760768.031096:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.031190:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764864 00000100:00100000:2.0:1789760768.031199:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764864 00000100:00100000:2.0:1789760768.031313:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746405452101 time 746450323338 state 0x0 ref 1 00000100:00100000:2.0:1789760768.031328:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746450583754 time 746450323338 state 0x1 ref 1 00000100:00100000:2.0:1789760768.031360:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746450583754 time 746496649124 state 0x0 ref 1 00000100:00100000:2.0:1789760768.031374:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515764864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.031419:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24449024, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.031429:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.031445:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760768.031450:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.031456:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.031474:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760768.040796:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760768.068657:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24449024 -> 24453120 00010000:00010000:2.0:1789760768.068691:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.068729:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.068732:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973286 is committed 00010000:00010000:2.0:1789760768.068739:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.068747:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760768.068752:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.068761:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.068781:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.068799:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.068897:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515764864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37521us (37805us total) trans 4294973286 rc 0/0 00000100:00100000:2.0:1789760768.068909:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5998 00000100:00100000:2.0:1789760768.068919:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746450583754 time 746496649124 state 0x2 ref 0 00000100:00100000:0.0:1789760768.074319:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.074461:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515764992 00000100:00100000:2.0:1789760768.074472:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515764992 00000100:00100000:2.0:1789760768.074526:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746450583754 time 746496649124 state 0x0 ref 1 00000100:00100000:2.0:1789760768.074542:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746496909540 time 746496649124 state 0x1 ref 1 00000100:00100000:2.0:1789760768.074571:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746496909540 time 746539859969 state 0x0 ref 1 00000100:00100000:2.0:1789760768.074582:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.074627:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24453120, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.074636:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.074651:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760768.074656:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.074662:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.074676:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.103379:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973287 is committed 00002000:00010000:2.0:1789760768.103387:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24453120 -> 24457216 00010000:00010000:2.0:1789760768.103403:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.103416:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.103426:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.103496:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760768.103502:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.103511:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.103530:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.103537:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.103676:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515764992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29090us (29361us total) trans 4294973287 rc 0/0 00000100:00100000:2.0:1789760768.103688:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 5999 00000100:00100000:2.0:1789760768.103698:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746496909540 time 746539859969 state 0x2 ref 0 00000100:00100000:1.0:1789760768.111621:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.112124:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765120 00000100:00100000:2.0:1789760768.112134:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765120 00000100:00100000:2.0:1789760768.112186:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746496909540 time 746539859969 state 0x0 ref 1 00000100:00100000:2.0:1789760768.112202:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746540120385 time 746539859969 state 0x1 ref 1 00000100:00100000:2.0:1789760768.112235:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746540120385 time 746577524527 state 0x0 ref 1 00000100:00100000:2.0:1789760768.112248:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.112367:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24457216, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.112375:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.112392:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760768.112397:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.112403:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.112418:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.143341:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973288 is committed 00002000:00010000:0.0:1789760768.143387:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24457216 -> 24461312 00010000:00010000:0.0:1789760768.143401:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.143415:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.143433:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.143502:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760768.143507:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.143525:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.143550:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.143558:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f5af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.143666:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31415us (32052us total) trans 4294973288 rc 0/0 00000100:00100000:0.0:1789760768.143676:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6000 00000100:00100000:0.0:1789760768.143687:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746540120385 time 746577524527 state 0x2 ref 0 00000100:00100000:3.0:1789760768.146147:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760768.146247:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765248 00000100:00100000:0.0:1789760768.146255:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765248 00000100:00100000:0.0:1789760768.146298:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746540120385 time 746577524527 state 0x0 ref 1 00000100:00100000:0.0:1789760768.146309:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746577784943 time 746577524527 state 0x1 ref 1 00000100:00100000:0.0:1789760768.146332:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746577784943 time 746611621926 state 0x0 ref 1 00000100:00100000:0.0:1789760768.146344:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760768.146391:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24461312, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760768.146399:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.146415:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760768.146420:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.146426:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.146438:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.172185:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973289 is committed 00002000:00010000:0.0:1789760768.172317:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24461312 -> 24465408 00010000:00010000:0.0:1789760768.172330:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.172339:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.172344:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.172349:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760768.172353:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.172359:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.172373:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.172381:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.172459:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26113us (26315us total) trans 4294973289 rc 0/0 00000100:00100000:0.0:1789760768.172470:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6001 00000100:00100000:0.0:1789760768.172479:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746577784943 time 746611621926 state 0x2 ref 0 00000100:00100000:3.0:1789760768.177454:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760768.177579:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765376 00000100:00100000:0.0:1789760768.177590:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765376 00000100:00100000:0.0:1789760768.177652:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746577784943 time 746611621926 state 0x0 ref 1 00000100:00100000:0.0:1789760768.177668:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746611882342 time 746611621926 state 0x1 ref 1 00000100:00100000:0.0:1789760768.177699:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746611882342 time 746642987924 state 0x0 ref 1 00000100:00100000:0.0:1789760768.177752:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515765376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760768.177813:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24465408, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760768.177822:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.177851:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760768.177856:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.177863:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.177881:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.225015:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973290 is committed 00002000:00010000:2.0:1789760768.225124:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24465408 -> 24469504 00010000:00010000:2.0:1789760768.225140:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.225155:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.225164:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.225172:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760768.225177:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.225186:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.225204:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.225211:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.225304:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515765376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47587us (47860us total) trans 4294973290 rc 0/0 00000100:00100000:2.0:1789760768.225313:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6002 00000100:00100000:2.0:1789760768.225323:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746611882342 time 746642987924 state 0x2 ref 0 00000100:00100000:1.0:1789760768.238331:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760768.238448:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765504 00000100:00100000:0.0:1789760768.238457:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765504 00000100:00100000:0.0:1789760768.238511:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746611882342 time 746642987924 state 0x0 ref 1 00000100:00100000:0.0:1789760768.238526:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746643248340 time 746642987924 state 0x1 ref 1 00000100:00100000:0.0:1789760768.238605:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746643248340 time 746703893705 state 0x0 ref 1 00000100:00100000:0.0:1789760768.238617:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515765504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760768.238668:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24469504, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760768.238676:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.238691:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760768.238696:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.238702:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.238753:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.282228:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973291 is committed 00002000:00010000:0.0:1789760768.282514:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24469504 -> 24473600 00010000:00010000:0.0:1789760768.282529:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.282541:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.282550:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.282556:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760768.282561:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.282570:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.282590:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.282597:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.282688:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515765504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44068us (44362us total) trans 4294973291 rc 0/0 00000100:00100000:0.0:1789760768.282861:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6003 00000100:00100000:0.0:1789760768.282870:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746643248340 time 746703893705 state 0x2 ref 0 00000100:00100000:3.0:1789760768.287320:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.287724:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765632 00000100:00100000:2.0:1789760768.287733:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765632 00000100:00100000:2.0:1789760768.287803:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746643248340 time 746703893705 state 0x0 ref 1 00000100:00100000:2.0:1789760768.287817:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746704154121 time 746703893705 state 0x1 ref 1 00000100:00100000:2.0:1789760768.287848:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746704154121 time 746753136755 state 0x0 ref 1 00000100:00100000:2.0:1789760768.287860:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.287906:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24473600, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.287915:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.287931:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760768.287935:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.287941:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.287956:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.320631:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760768.339634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973292 is committed 00002000:00010000:3.0:1789760768.339732:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24473600 -> 24477696 00010000:00010000:3.0:1789760768.339748:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.339761:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.339770:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.339778:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760768.339793:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.339802:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.339827:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.339835:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f5cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.339920:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52057us (52697us total) trans 4294973292 rc 0/0 00000100:00100000:3.0:1789760768.339930:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6004 00000100:00100000:3.0:1789760768.339940:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746704154121 time 746753136755 state 0x2 ref 0 00000100:00100000:3.0:1789760768.343396:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.343529:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765760 00000100:00100000:2.0:1789760768.343539:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765760 00000100:00100000:2.0:1789760768.343588:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746704154121 time 746753136755 state 0x0 ref 1 00000100:00100000:2.0:1789760768.343603:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746753397171 time 746753136755 state 0x1 ref 1 00000100:00100000:2.0:1789760768.343636:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746753397171 time 746808925441 state 0x0 ref 1 00000100:00100000:2.0:1789760768.343652:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515765760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.343697:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24477696, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.343739:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.343755:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760768.343760:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.343765:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.343778:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.373855:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973293 is committed 00002000:00010000:2.0:1789760768.373995:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24477696 -> 24481792 00010000:00010000:2.0:1789760768.374008:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.374022:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.374031:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.374038:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760768.374043:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.374053:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.374074:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.374082:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f5d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.374372:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515765760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30717us (30980us total) trans 4294973293 rc 0/0 00000100:00100000:2.0:1789760768.374382:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6005 00000100:00100000:2.0:1789760768.374392:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746753397171 time 746808925441 state 0x2 ref 0 00000100:00100000:2.0:1789760768.379012:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760768.379113:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515765888 00000100:00100000:0.0:1789760768.379123:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515765888 00000100:00100000:0.0:1789760768.379308:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746753397171 time 746808925441 state 0x0 ref 1 00000100:00100000:0.0:1789760768.379323:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746809185857 time 746808925441 state 0x1 ref 1 00000100:00100000:0.0:1789760768.379390:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746809185857 time 746844678115 state 0x0 ref 1 00000100:00100000:0.0:1789760768.379403:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515765888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760768.379520:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24481792, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760768.379529:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.379546:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760768.379550:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.379556:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.379569:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.415321:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24481792 -> 24485888 00010000:00010000:1.0:1789760768.415340:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.415353:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.415362:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.415375:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760768.415380:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.415389:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.415407:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.415415:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f5d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.415522:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515765888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36116us (36515us total) trans 4294973294 rc 0/0 00000100:00100000:1.0:1789760768.415532:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6006 00000100:00100000:1.0:1789760768.415542:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746809185857 time 746844678115 state 0x2 ref 0 00000001:00080000:3.0:1789760768.415679:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973294 is committed 00000100:00100000:3.0:1789760768.418802:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760768.419067:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766016 00000100:00100000:1.0:1789760768.419077:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766016 00000100:00100000:1.0:1789760768.419141:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746809185857 time 746844678115 state 0x0 ref 1 00000100:00100000:1.0:1789760768.419156:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746844938531 time 746844678115 state 0x1 ref 1 00000100:00100000:1.0:1789760768.419179:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746844938531 time 746884467912 state 0x0 ref 1 00000100:00100000:1.0:1789760768.419195:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515766016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760768.419241:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24485888, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:1.0:1789760768.419251:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.419266:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760768.419271:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.419277:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.419291:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.450324:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24485888 -> 24489984 00010000:00010000:1.0:1789760768.450344:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450357:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450367:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450374:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760768.450379:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450388:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450406:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.450413:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f5e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.450517:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515766016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31320us (31719us total) trans 4294973295 rc 0/0 00000100:00100000:1.0:1789760768.450528:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6007 00000100:00100000:1.0:1789760768.450537:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746844938531 time 746884467912 state 0x2 ref 0 00000100:00100000:3.0:1789760768.454257:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760768.454614:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766144 00000100:00100000:1.0:1789760768.454626:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766144 00000100:00100000:1.0:1789760768.454680:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746844938531 time 746884467912 state 0x0 ref 1 00000100:00100000:1.0:1789760768.454696:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746884728328 time 746884467912 state 0x1 ref 1 00000100:00100000:1.0:1789760768.454750:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746884728328 time 746920038967 state 0x0 ref 1 00000100:00100000:1.0:1789760768.454761:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515766144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760768.454820:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24489984, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:1.0:1789760768.454829:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.454843:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760768.454847:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.454853:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.454865:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.457653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973295 is committed 00002000:00010000:1.0:1789760768.480935:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24489984 -> 24494080 00010000:00010000:1.0:1789760768.480953:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.480966:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.480975:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.480982:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760768.480986:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.480994:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481014:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481023:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f5e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.481114:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515766144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26350us (26862us total) trans 4294973296 rc 0/0 00000100:00100000:1.0:1789760768.481125:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6008 00000100:00100000:1.0:1789760768.481136:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746884728328 time 746920038967 state 0x2 ref 0 00000001:00080000:3.0:1789760768.482333:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973296 is committed 00000100:00100000:3.0:1789760768.486194:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760768.486272:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766272 00000100:00100000:0.0:1789760768.486282:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766272 00000100:00100000:0.0:1789760768.486332:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746884728328 time 746920038967 state 0x0 ref 1 00000100:00100000:0.0:1789760768.486346:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746920299383 time 746920038967 state 0x1 ref 1 00000100:00100000:0.0:1789760768.486374:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746920299383 time 746951663781 state 0x0 ref 1 00000100:00100000:0.0:1789760768.486386:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515766272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760768.486437:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24494080, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760768.486456:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486472:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760768.486476:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.486482:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486495:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.527955:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24494080 -> 24498176 00010000:00010000:1.0:1789760768.527977:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.527990:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.528001:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.528008:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760768.528013:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.528022:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.528046:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.528054:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1f5ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.528318:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515766272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41929us (42128us total) trans 4294973297 rc 0/0 00000100:00100000:1.0:1789760768.528329:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6009 00000100:00100000:1.0:1789760768.528338:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746920299383 time 746951663781 state 0x2 ref 0 00000001:00080000:2.0:1789760768.528987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973297 is committed 00000100:00100000:3.0:1789760768.540529:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.540866:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766400 00000100:00100000:2.0:1789760768.540876:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766400 00000100:00100000:2.0:1789760768.540930:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746920299383 time 746951663781 state 0x0 ref 1 00000100:00100000:2.0:1789760768.540946:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746951924197 time 746951663781 state 0x1 ref 1 00000100:00100000:2.0:1789760768.540978:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746951924197 time 747006266763 state 0x0 ref 1 00000100:00100000:2.0:1789760768.540991:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cab450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515766400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.541038:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24498176, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.541048:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.541063:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.541068:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.541188:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.541207:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.594459:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973298 is committed 00002000:00010000:1.0:1789760768.594580:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24498176 -> 24502272 00010000:00010000:1.0:1789760768.594605:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.594621:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.594631:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.594639:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760768.594645:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.594661:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.594687:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.594696:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f5f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.595245:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cab450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515766400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54249us (54718us total) trans 4294973298 rc 0/0 00000100:00100000:1.0:1789760768.595257:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6010 00000100:00100000:1.0:1789760768.595270:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746951924197 time 747006266763 state 0x2 ref 0 00000100:00100000:3.0:1789760768.603679:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.605105:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766528 00000100:00100000:2.0:1789760768.605125:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766528 00000100:00100000:2.0:1789760768.605214:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 746951924197 time 747006266763 state 0x0 ref 1 00000100:00100000:2.0:1789760768.605234:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747006527179 time 747006266763 state 0x1 ref 1 00000100:00100000:2.0:1789760768.605344:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747006527179 time 747070631853 state 0x0 ref 1 00000100:00100000:2.0:1789760768.605361:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515766528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.605443:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24502272, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.605454:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.605475:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.605481:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.605489:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.605508:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.679056:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24502272 -> 24506368 00010000:00010000:3.0:1789760768.679081:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.679095:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.679110:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.679118:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760768.679123:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.679132:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.679152:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.679160:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f5fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.679249:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515766528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73888us (75581us total) trans 4294973299 rc 0/0 00000100:00100000:3.0:1789760768.679260:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6011 00000100:00100000:3.0:1789760768.679269:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747006527179 time 747070631853 state 0x2 ref 0 00000001:00080000:2.0:1789760768.679370:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973299 is committed 00000100:00100000:3.0:1789760768.682759:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760768.683033:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766656 00000100:00100000:1.0:1789760768.683048:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766656 00000100:00100000:1.0:1789760768.683122:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747006527179 time 747070631853 state 0x0 ref 1 00000100:00100000:1.0:1789760768.683142:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747070892269 time 747070631853 state 0x1 ref 1 00000100:00100000:1.0:1789760768.683293:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747070892269 time 747148579390 state 0x0 ref 1 00000100:00100000:1.0:1789760768.683314:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515766656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760768.683378:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24506368, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:1.0:1789760768.683390:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.683412:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.683417:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.683425:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.683442:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.741416:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24506368 -> 24510464 00010000:00010000:2.0:1789760768.741562:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.741578:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.741587:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.741595:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760768.741600:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.741610:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.741636:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.741649:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f603 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.742091:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515766656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58775us (59393us total) trans 4294973300 rc 0/0 00000100:00100000:2.0:1789760768.742163:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6012 00000100:00100000:2.0:1789760768.742176:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747070892269 time 747148579390 state 0x2 ref 0 00000001:00080000:3.0:1789760768.743205:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973300 is committed 00000100:00100000:1.0:1789760768.745689:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760768.745977:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766784 00000100:00100000:0.0:1789760768.745992:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766784 00000100:00100000:0.0:1789760768.746069:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747070892269 time 747148579390 state 0x0 ref 1 00000100:00100000:0.0:1789760768.746088:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747148839806 time 747148579390 state 0x1 ref 1 00000100:00100000:0.0:1789760768.746203:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747148839806 time 747211490038 state 0x0 ref 1 00000100:00100000:0.0:1789760768.746220:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515766784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760768.746296:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24510464, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:0.0:1789760768.746305:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.746323:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.746327:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.746333:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.746346:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.799018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973301 is committed 00002000:00010000:2.0:1789760768.799285:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24510464 -> 24514560 00010000:00010000:2.0:1789760768.799317:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.799332:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.799342:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.799351:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760768.799357:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.799367:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.799401:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.799410:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f60a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.799851:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515766784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53478us (54017us total) trans 4294973301 rc 0/0 00000100:00100000:2.0:1789760768.799865:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6013 00000100:00100000:2.0:1789760768.799877:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747148839806 time 747211490038 state 0x2 ref 0 00000100:00100000:3.0:1789760768.804353:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760768.804473:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515766912 00000100:00100000:1.0:1789760768.804487:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515766912 00000100:00100000:1.0:1789760768.804560:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747148839806 time 747211490038 state 0x0 ref 1 00000100:00100000:1.0:1789760768.804579:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747211750454 time 747211490038 state 0x1 ref 1 00000100:00100000:1.0:1789760768.804615:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747211750454 time 747269903537 state 0x0 ref 1 00000100:00100000:1.0:1789760768.804630:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515766912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760768.804699:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24514560, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:1.0:1789760768.804747:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.804766:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.804772:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.804790:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.804807:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.889299:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24514560 -> 24518656 00010000:00010000:3.0:1789760768.889328:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.889346:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.889356:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.889365:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760768.889371:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.889382:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.889409:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.889418:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f611 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.889548:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515766912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 84913us (85201us total) trans 4294973302 rc 0/0 00000100:00100000:3.0:1789760768.889560:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6014 00000100:00100000:3.0:1789760768.889572:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747211750454 time 747269903537 state 0x2 ref 0 00000001:00080000:0.0:1789760768.889663:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973302 is committed 00000100:00100000:3.0:1789760768.892682:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.892933:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767040 00000100:00100000:2.0:1789760768.892947:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767040 00000100:00100000:2.0:1789760768.893087:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747211750454 time 747269903537 state 0x0 ref 1 00000100:00100000:2.0:1789760768.893109:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747270163953 time 747269903537 state 0x1 ref 1 00000100:00100000:2.0:1789760768.893148:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747270163953 time 747358436952 state 0x0 ref 1 00000100:00100000:2.0:1789760768.893162:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515767040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.893227:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24518656, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc007 00010000:00010000:2.0:1789760768.893237:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893257:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.893263:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.893270:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893287:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.960317:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24518656 -> 24522752 00002000:00010000:1.0:1789760768.960330:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:1.0:1789760768.960333:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00010000:00010000:1.0:1789760768.960352:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.960367:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.960378:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.960387:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760768.960392:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.960403:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.960428:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.960443:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f618 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.960566:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515767040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 67399us (67891us total) trans 4294973303 rc 0/0 00000100:00100000:1.0:1789760768.960580:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6015 00000100:00100000:1.0:1789760768.960592:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747270163953 time 747358436952 state 0x2 ref 0 00000001:00080000:3.0:1789760768.960681:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973303 is committed 00000100:00100000:1.0:1789760768.965910:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760768.966079:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767168 00000100:00100000:2.0:1789760768.966092:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767168 00000100:00100000:2.0:1789760768.966173:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747270163953 time 747358436952 state 0x0 ref 1 00000100:00100000:2.0:1789760768.966194:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747358697368 time 747358436952 state 0x1 ref 1 00000100:00100000:2.0:1789760768.966230:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747358697368 time 747431519135 state 0x0 ref 1 00000100:00100000:2.0:1789760768.966245:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760768.966312:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24522752, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760768.966324:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.966344:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.966348:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.966357:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.966624:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.014633:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973304 is committed 00002000:00010000:3.0:1789760769.014789:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24522752 -> 24526848 00010000:00010000:3.0:1789760769.014810:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.014826:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.014837:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.014846:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760769.014851:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.014862:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.014886:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.014893:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f61f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.015058:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48807us (49157us total) trans 4294973304 rc 0/0 00000100:00100000:3.0:1789760769.015071:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6016 00000100:00100000:3.0:1789760769.015082:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747358697368 time 747431519135 state 0x2 ref 0 00000100:00100000:0.0:1789760769.024351:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760769.024573:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767296 00000100:00100000:3.0:1789760769.024586:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767296 00000100:00100000:3.0:1789760769.024657:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747358697368 time 747431519135 state 0x0 ref 1 00000100:00100000:3.0:1789760769.024676:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747431779551 time 747431519135 state 0x1 ref 1 00000100:00100000:3.0:1789760769.024807:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747431779551 time 747490094949 state 0x0 ref 1 00000100:00100000:3.0:1789760769.024821:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515767296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760769.024887:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24526848, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:3.0:1789760769.024898:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.024917:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760769.024922:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.024929:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.024944:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.061092:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973305 is committed 00002000:00010000:1.0:1789760769.061318:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24526848 -> 24530944 00010000:00010000:1.0:1789760769.061338:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.061353:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.061363:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.061371:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.061376:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.061385:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.061414:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.061422:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f626 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.061564:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515767296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36737us (37220us total) trans 4294973305 rc 0/0 00000100:00100000:1.0:1789760769.061576:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6017 00000100:00100000:1.0:1789760769.061589:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747431779551 time 747490094949 state 0x2 ref 0 00000100:00100000:1.0:1789760769.065947:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.066191:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767424 00000100:00100000:2.0:1789760769.066205:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767424 00000100:00100000:2.0:1789760769.066280:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747431779551 time 747490094949 state 0x0 ref 1 00000100:00100000:2.0:1789760769.066299:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747490355365 time 747490094949 state 0x1 ref 1 00000100:00100000:2.0:1789760769.066334:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747490355365 time 747531622759 state 0x0 ref 1 00000100:00100000:2.0:1789760769.066347:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.066417:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24530944, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760769.066428:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.066451:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760769.066456:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.066464:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.066481:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.096686:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24530944 -> 24535040 00010000:00010000:2.0:1789760769.096746:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.096770:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.096790:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.096798:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.096804:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.096815:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.096839:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.096848:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1f62d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.096964:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515767424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30611us (31027us total) trans 4294973306 rc 0/0 00000100:00100000:2.0:1789760769.096977:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6018 00000100:00100000:2.0:1789760769.096989:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747490355365 time 747531622759 state 0x2 ref 0 00000001:00080000:0.0:1789760769.097431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973306 is committed 00000100:00100000:2.0:1789760769.101351:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.101477:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767552 00000100:00100000:1.0:1789760769.101492:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767552 00000100:00100000:1.0:1789760769.101568:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747490355365 time 747531622759 state 0x0 ref 1 00000100:00100000:1.0:1789760769.101587:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747531883175 time 747531622759 state 0x1 ref 1 00000100:00100000:1.0:1789760769.101618:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747531883175 time 747566907437 state 0x0 ref 1 00000100:00100000:1.0:1789760769.101632:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515767552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.102096:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24535040, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.102109:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.102136:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760769.102142:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.102150:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.102167:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.145351:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973307 is committed 00002000:00010000:1.0:1789760769.145481:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24535040 -> 24539136 00010000:00010000:1.0:1789760769.145502:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.145518:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.145527:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.145536:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.145541:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.145552:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.145580:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.145588:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f634 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.145867:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515767552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44227us (44522us total) trans 4294973307 rc 0/0 00000100:00100000:1.0:1789760769.145880:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6019 00000100:00100000:1.0:1789760769.145893:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747531883175 time 747566907437 state 0x2 ref 0 00000100:00100000:1.0:1789760769.151281:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760769.151384:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767680 00000100:00100000:3.0:1789760769.151398:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767680 00000100:00100000:3.0:1789760769.151485:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747531883175 time 747566907437 state 0x0 ref 1 00000100:00100000:3.0:1789760769.151506:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747567167853 time 747566907437 state 0x1 ref 1 00000100:00100000:3.0:1789760769.151579:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747567167853 time 747616867219 state 0x0 ref 1 00000100:00100000:3.0:1789760769.151593:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760769.151657:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24539136, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:3.0:1789760769.151668:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.151690:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760769.151745:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.151754:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.151772:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.185275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973308 is committed 00002000:00010000:0.0:1789760769.185664:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24539136 -> 24543232 00010000:00010000:0.0:1789760769.185685:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.185754:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.185765:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.185790:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760769.185796:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.185806:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.185840:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.185848:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f63b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.187520:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35913us (36237us total) trans 4294973308 rc 0/0 00000100:00100000:1.0:1789760769.187947:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6020 00000100:00100000:1.0:1789760769.187961:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747567167853 time 747616867219 state 0x2 ref 0 00000100:00100000:2.0:1789760769.193097:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.193233:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767808 00000100:00100000:1.0:1789760769.193249:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767808 00000100:00100000:1.0:1789760769.193309:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747567167853 time 747616867219 state 0x0 ref 1 00000100:00100000:1.0:1789760769.193325:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747617127635 time 747616867219 state 0x1 ref 1 00000100:00100000:1.0:1789760769.193361:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747617127635 time 747658649929 state 0x0 ref 1 00000100:00100000:1.0:1789760769.193374:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515767808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.193423:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24543232, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.193437:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.193453:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760769.193458:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.193464:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.193478:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.227471:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24543232 -> 24547328 00010000:00010000:1.0:1789760769.227499:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.227514:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.227524:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.227532:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.227537:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.227546:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.227577:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.227585:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f642 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.227757:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515767808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34377us (34670us total) trans 4294973309 rc 0/0 00000100:00100000:1.0:1789760769.227787:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6021 00000100:00100000:1.0:1789760769.227800:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747617127635 time 747658649929 state 0x2 ref 0 00000001:00080000:0.0:1789760769.228476:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973309 is committed 00000100:00100000:2.0:1789760769.233403:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760769.234051:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515767936 00000100:00100000:3.0:1789760769.234066:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515767936 00000100:00100000:3.0:1789760769.234135:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747617127635 time 747658649929 state 0x0 ref 1 00000100:00100000:3.0:1789760769.234153:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747658910345 time 747658649929 state 0x1 ref 1 00000100:00100000:3.0:1789760769.234246:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747658910345 time 747699534479 state 0x0 ref 1 00000100:00100000:3.0:1789760769.234261:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760769.234326:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24547328, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:3.0:1789760769.234337:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.234357:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760769.234362:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.234369:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.234386:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.284417:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973310 is committed 00002000:00010000:0.0:1789760769.284552:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24547328 -> 24551424 00010000:00010000:0.0:1789760769.284573:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.284589:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.284599:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.284608:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760769.284614:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.284624:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.284649:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.284658:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f649 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.285094:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515767936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50828us (51696us total) trans 4294973310 rc 0/0 00000100:00100000:0.0:1789760769.285108:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6022 00000100:00100000:0.0:1789760769.285120:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747658910345 time 747699534479 state 0x2 ref 0 00000100:00100000:3.0:1789760769.288453:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.288632:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768064 00000100:00100000:1.0:1789760769.288646:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768064 00000100:00100000:1.0:1789760769.289029:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747658910345 time 747699534479 state 0x0 ref 1 00000100:00100000:1.0:1789760769.289050:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747699794895 time 747699534479 state 0x1 ref 1 00000100:00100000:1.0:1789760769.289171:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747699794895 time 747754458217 state 0x0 ref 1 00000100:00100000:1.0:1789760769.289189:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515768064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.289258:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24551424, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.289268:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.289293:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760769.289297:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.289305:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.289321:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.341913:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760769.348539:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973311 is committed 00002000:00010000:2.0:1789760769.348816:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24551424 -> 24555520 00010000:00010000:2.0:1789760769.348839:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.348856:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.348866:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.348875:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.348881:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.348891:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.348917:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.348926:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f650 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.349046:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515768064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59854us (60598us total) trans 4294973311 rc 0/0 00000100:00100000:2.0:1789760769.349059:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6023 00000100:00100000:2.0:1789760769.349072:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747699794895 time 747754458217 state 0x2 ref 0 00000100:00100000:2.0:1789760769.352772:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.352940:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768192 00000100:00100000:1.0:1789760769.352954:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768192 00000100:00100000:1.0:1789760769.353029:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747699794895 time 747754458217 state 0x0 ref 1 00000100:00100000:1.0:1789760769.353049:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747754718633 time 747754458217 state 0x1 ref 1 00000100:00100000:1.0:1789760769.353084:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747754718633 time 747818373149 state 0x0 ref 1 00000100:00100000:1.0:1789760769.353099:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515768192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.353168:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24555520, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.353179:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.353200:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760769.353206:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.353214:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.353231:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.400140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973312 is committed 00002000:00010000:1.0:1789760769.400449:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24555520 -> 24559616 00010000:00010000:1.0:1789760769.400479:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.400493:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.400619:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.400632:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.400638:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.400649:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.400674:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.400684:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f657 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.400900:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515768192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47797us (48135us total) trans 4294973312 rc 0/0 00000100:00100000:1.0:1789760769.400913:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6024 00000100:00100000:1.0:1789760769.400925:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747754718633 time 747818373149 state 0x2 ref 0 00000100:00100000:1.0:1789760769.407003:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.407259:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768320 00000100:00100000:2.0:1789760769.407274:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768320 00000100:00100000:2.0:1789760769.407357:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747754718633 time 747818373149 state 0x0 ref 1 00000100:00100000:2.0:1789760769.407376:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747818633565 time 747818373149 state 0x1 ref 1 00000100:00100000:2.0:1789760769.407698:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747818633565 time 747872700456 state 0x0 ref 1 00000100:00100000:2.0:1789760769.407750:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.407833:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24559616, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760769.407843:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.407864:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760769.407868:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.407883:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.407899:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.476430:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973313 is committed 00002000:00010000:1.0:1789760769.477454:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24559616 -> 24563712 00010000:00010000:1.0:1789760769.477478:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.477494:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.477503:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.477512:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.477518:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.477528:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.477554:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.477569:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f65e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.477702:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69953us (70709us total) trans 4294973313 rc 0/0 00000100:00100000:1.0:1789760769.477769:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6025 00000100:00100000:1.0:1789760769.477791:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747818633565 time 747872700456 state 0x2 ref 0 00000100:00100000:3.0:1789760769.483437:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.483654:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768448 00000100:00100000:1.0:1789760769.483668:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768448 00000100:00100000:1.0:1789760769.483797:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747818633565 time 747872700456 state 0x0 ref 1 00000100:00100000:1.0:1789760769.483816:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747872960872 time 747872700456 state 0x1 ref 1 00000100:00100000:1.0:1789760769.483862:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747872960872 time 747949150438 state 0x0 ref 1 00000100:00100000:1.0:1789760769.483877:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.483943:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24563712, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.483954:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.483974:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760769.483980:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.483988:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.484011:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.534493:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760769.534532:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544417792:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760769.534619:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.534743:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417792 00000100:00100000:0.0:1789760769.534754:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417792 00000100:00080000:1.0:1789760769.534878:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760769.535178:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789760769.535192:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544417792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760769.535240:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760769.535299:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544417792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 110us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.535307:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:1.0:1789760769.536637:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c58000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544417792:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760769.537469:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544417920:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760769.537760:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.537846:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544417920 00000100:00100000:2.0:1789760769.537858:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544417920 00000100:00100000:2.0:1789760769.537922:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1789760769.537931:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544417920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760769.537972:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3833593856 avail 3831496704 00000100:00100000:2.0:1789760769.538021:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544417920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.538029:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:2.0:1789760769.538079:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c63800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544417920:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760769.540766:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760769.553950:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24563712 -> 24567808 00010000:00010000:1.0:1789760769.553982:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.553997:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.554006:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.554014:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.554020:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.554031:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.554065:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.554073:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f665 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.554277:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515768448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70321us (70785us total) trans 4294973314 rc 0/0 00000100:00100000:1.0:1789760769.554292:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6026 00000100:00100000:1.0:1789760769.554305:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747872960872 time 747949150438 state 0x2 ref 0 00000001:00080000:2.0:1789760769.554588:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973314 is committed 00000100:00100000:2.0:1789760769.557892:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.558371:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768576 00000100:00100000:1.0:1789760769.558384:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768576 00000100:00100000:1.0:1789760769.558479:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747872960872 time 747949150438 state 0x0 ref 1 00000100:00100000:1.0:1789760769.558498:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747949410854 time 747949150438 state 0x1 ref 1 00000100:00100000:1.0:1789760769.558904:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747949410854 time 748024191044 state 0x0 ref 1 00000100:00100000:1.0:1789760769.558921:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515768576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.558989:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24567808, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.559000:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.559022:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.559028:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.559036:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.559053:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.603109:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973315 is committed 00002000:00010000:1.0:1789760769.603323:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24567808 -> 24571904 00010000:00010000:1.0:1789760769.603346:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.603362:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.603373:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.603382:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.603388:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.603399:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.603425:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.603439:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f66c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.603649:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515768576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44724us (45766us total) trans 4294973315 rc 0/0 00000100:00100000:1.0:1789760769.603661:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6027 00000100:00100000:1.0:1789760769.603674:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747949410854 time 748024191044 state 0x2 ref 0 00000100:00100000:3.0:1789760769.606193:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760769.606373:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768704 00000100:00100000:0.0:1789760769.606387:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768704 00000100:00100000:0.0:1789760769.606468:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 747949410854 time 748024191044 state 0x0 ref 1 00000100:00100000:0.0:1789760769.606488:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748024451460 time 748024191044 state 0x1 ref 1 00000100:00100000:0.0:1789760769.606624:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748024451460 time 748071911509 state 0x0 ref 1 00000100:00100000:0.0:1789760769.606645:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760769.606765:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24571904, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:0.0:1789760769.606786:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.606807:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.606813:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.606821:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.606837:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.648533:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973316 is committed 00002000:00010000:3.0:1789760769.648809:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24571904 -> 24576000 00010000:00010000:3.0:1789760769.648830:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.648846:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.648856:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.648865:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760769.648873:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.648885:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.648911:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.648921:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f673 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.649092:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42446us (42905us total) trans 4294973316 rc 0/0 00000100:00100000:3.0:1789760769.649106:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6028 00000100:00100000:3.0:1789760769.649120:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748024451460 time 748071911509 state 0x2 ref 0 00000100:00100000:3.0:1789760769.653355:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.653522:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768832 00000100:00100000:2.0:1789760769.653537:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768832 00000100:00100000:2.0:1789760769.653615:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748024451460 time 748071911509 state 0x0 ref 1 00000100:00100000:2.0:1789760769.653636:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748072171925 time 748071911509 state 0x1 ref 1 00000100:00100000:2.0:1789760769.653677:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748072171925 time 748118965529 state 0x0 ref 1 00000100:00100000:2.0:1789760769.653691:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.653817:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24576000, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760769.653830:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653852:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.653858:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.653866:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653882:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.693227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973317 is committed 00002000:00010000:2.0:1789760769.693589:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24576000 -> 24580096 00010000:00010000:2.0:1789760769.693611:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.693626:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.693637:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.693645:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.693651:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.693661:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.693688:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.693744:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f67a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.693909:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515768832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40212us (40561us total) trans 4294973317 rc 0/0 00000100:00100000:2.0:1789760769.693921:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6029 00000100:00100000:2.0:1789760769.693934:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748072171925 time 748118965529 state 0x2 ref 0 00000100:00100000:1.0:1789760769.697630:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.697876:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515768960 00000100:00100000:2.0:1789760769.697890:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515768960 00000100:00100000:2.0:1789760769.697969:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748072171925 time 748118965529 state 0x0 ref 1 00000100:00100000:2.0:1789760769.697989:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748119225945 time 748118965529 state 0x1 ref 1 00000100:00100000:2.0:1789760769.698027:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748119225945 time 748163316025 state 0x0 ref 1 00000100:00100000:2.0:1789760769.698043:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515768960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.698112:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24580096, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760769.698123:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.698142:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.698148:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.698155:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.698177:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.736244:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24580096 -> 24584192 00010000:00010000:2.0:1789760769.736271:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.736286:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.736296:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.736304:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.736310:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.736320:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.736343:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.736352:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f681 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.736486:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515768960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38439us (38876us total) trans 4294973318 rc 0/0 00000100:00100000:2.0:1789760769.736500:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6030 00000100:00100000:2.0:1789760769.736512:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748119225945 time 748163316025 state 0x2 ref 0 00000001:00080000:3.0:1789760769.736644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973318 is committed 00000100:00100000:1.0:1789760769.740073:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.740160:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769088 00000100:00100000:2.0:1789760769.740170:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769088 00000100:00100000:2.0:1789760769.740236:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748119225945 time 748163316025 state 0x0 ref 1 00000100:00100000:2.0:1789760769.740255:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748163576441 time 748163316025 state 0x1 ref 1 00000100:00100000:2.0:1789760769.740321:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748163576441 time 748205609032 state 0x0 ref 1 00000100:00100000:2.0:1789760769.740336:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515769088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.740399:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24584192, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760769.740662:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.740681:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.740685:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.740787:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.740806:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.795121:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973319 is committed 00002000:00010000:2.0:1789760769.795270:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24584192 -> 24588288 00010000:00010000:2.0:1789760769.795293:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.795311:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.795324:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.795334:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.795339:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.795354:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.795387:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.795397:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f688 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.795645:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515769088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55304us (55581us total) trans 4294973319 rc 0/0 00000100:00100000:2.0:1789760769.795662:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6031 00000100:00100000:2.0:1789760769.795675:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748163576441 time 748205609032 state 0x2 ref 0 00000100:00100000:1.0:1789760769.799026:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.799142:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769216 00000100:00100000:2.0:1789760769.799155:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769216 00000100:00100000:2.0:1789760769.799225:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748163576441 time 748205609032 state 0x0 ref 1 00000100:00100000:2.0:1789760769.799245:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748205869448 time 748205609032 state 0x1 ref 1 00000100:00100000:2.0:1789760769.799305:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748205869448 time 748264593272 state 0x0 ref 1 00000100:00100000:2.0:1789760769.799318:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515769216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.799383:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24588288, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:2.0:1789760769.799395:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.799470:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.799476:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.799484:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.799503:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.851368:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973320 is committed 00002000:00010000:1.0:1789760769.851959:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24588288 -> 24592384 00010000:00010000:1.0:1789760769.851982:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.852000:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.852010:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.852018:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760769.852024:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.852041:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.852070:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.852079:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f68f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.854540:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515769216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55205us (55510us total) trans 4294973320 rc 0/0 00000100:00100000:1.0:1789760769.854560:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6032 00000100:00100000:1.0:1789760769.854576:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748205869448 time 748264593272 state 0x2 ref 0 00000100:00100000:3.0:1789760769.862033:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760769.863766:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769344 00000100:00100000:1.0:1789760769.863796:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769344 00000100:00100000:1.0:1789760769.863888:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748205869448 time 748264593272 state 0x0 ref 1 00000100:00100000:1.0:1789760769.863908:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748264853688 time 748264593272 state 0x1 ref 1 00000100:00100000:1.0:1789760769.863952:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748264853688 time 748329240828 state 0x0 ref 1 00000100:00100000:1.0:1789760769.863967:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515769344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760769.864035:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24592384, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xc007 00010000:00010000:1.0:1789760769.864046:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.864068:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.864073:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.864081:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.864097:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.925098:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24592384 -> 24596480 00002000:00010000:2.0:1789760769.925109:0:13071:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760769 -> 1789760770 00002000:00010000:2.0:1789760769.925111:0:13071:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760769 -> 1789760770 00010000:00010000:2.0:1789760769.925124:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.925137:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.925145:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.925152:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.925157:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.925165:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.925184:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.925191:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f696 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.925304:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973321 is committed 00000100:00100000:2.0:1789760769.925335:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515769344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61365us (63313us total) trans 4294973321 rc 0/0 00000100:00100000:2.0:1789760769.925348:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6033 00000100:00100000:2.0:1789760769.925358:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748264853688 time 748329240828 state 0x2 ref 0 00000100:00100000:2.0:1789760769.934080:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760769.934221:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769472 00000100:00100000:3.0:1789760769.934236:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769472 00000100:00100000:3.0:1789760769.934314:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748264853688 time 748329240828 state 0x0 ref 1 00000100:00100000:3.0:1789760769.934345:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748329501244 time 748329240828 state 0x1 ref 1 00000100:00100000:3.0:1789760769.934463:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748329501244 time 748399750615 state 0x0 ref 1 00000100:00100000:3.0:1789760769.934479:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515769472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760769.934547:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24596480, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:3.0:1789760769.934560:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.934580:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.934585:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.934593:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.934611:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.988629:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973322 is committed 00002000:00010000:2.0:1789760769.989010:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24596480 -> 24600576 00010000:00010000:2.0:1789760769.989024:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.989034:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.989040:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.989045:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760769.989049:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.989055:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.989070:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.989074:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1f69d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.989212:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515769472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54732us (55142us total) trans 4294973322 rc 0/0 00000100:00100000:2.0:1789760769.989223:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6034 00000100:00100000:2.0:1789760769.989232:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748329501244 time 748399750615 state 0x2 ref 0 00000100:00100000:1.0:1789760769.997418:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760769.997508:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769600 00000100:00100000:2.0:1789760769.997519:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769600 00000100:00100000:2.0:1789760769.997574:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748329501244 time 748399750615 state 0x0 ref 1 00000100:00100000:2.0:1789760769.997588:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748400011031 time 748399750615 state 0x1 ref 1 00000100:00100000:2.0:1789760769.997616:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748400011031 time 748462905592 state 0x0 ref 1 00000100:00100000:2.0:1789760769.997629:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515769600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760769.997680:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24600576, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:2.0:1789760769.997690:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.997752:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.997756:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.997762:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.997785:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.037090:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973323 is committed 00002000:00010000:2.0:1789760770.037189:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24600576 -> 24604672 00010000:00010000:2.0:1789760770.037204:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.037216:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.037225:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.037232:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760770.037237:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.037246:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.037264:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.037272:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.037403:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515769600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39771us (39989us total) trans 4294973323 rc 0/0 00000100:00100000:2.0:1789760770.037478:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6035 00000100:00100000:2.0:1789760770.037488:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748400011031 time 748462905592 state 0x2 ref 0 00000100:00100000:3.0:1789760770.043602:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.044001:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769728 00000100:00100000:1.0:1789760770.044012:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769728 00000100:00100000:1.0:1789760770.044087:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748400011031 time 748462905592 state 0x0 ref 1 00000100:00100000:1.0:1789760770.044103:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748463166008 time 748462905592 state 0x1 ref 1 00000100:00100000:1.0:1789760770.044179:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748463166008 time 748509468252 state 0x0 ref 1 00000100:00100000:1.0:1789760770.044193:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515769728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.044252:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24604672, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:1.0:1789760770.044261:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.044278:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760770.044282:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00000100:00080000:3.0:1789760770.044523:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760770.044613:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.044629:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.089701:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973324 is committed 00002000:00010000:0.0:1789760770.090277:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24604672 -> 24608768 00010000:00010000:0.0:1789760770.090293:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.090306:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.090316:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.090324:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760770.090329:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.090338:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.090357:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.090365:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f6ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.090555:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515769728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46356us (46953us total) trans 4294973324 rc 0/0 00000100:00100000:0.0:1789760770.090573:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6036 00000100:00100000:0.0:1789760770.090583:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748463166008 time 748509468252 state 0x2 ref 0 00000100:00100000:1.0:1789760770.094174:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760770.094264:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769856 00000100:00100000:2.0:1789760770.094274:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769856 00000100:00100000:2.0:1789760770.094322:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748463166008 time 748509468252 state 0x0 ref 1 00000100:00100000:2.0:1789760770.094335:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748509728668 time 748509468252 state 0x1 ref 1 00000100:00100000:2.0:1789760770.094403:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748509728668 time 748559692217 state 0x0 ref 1 00000100:00100000:2.0:1789760770.094422:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515769856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760770.094469:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24608768, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:2.0:1789760770.094478:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.094493:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760770.094498:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.094504:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.094518:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.152337:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973325 is committed 00002000:00010000:1.0:1789760770.153117:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24608768 -> 24612864 00010000:00010000:1.0:1789760770.153134:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.153147:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.153156:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.153163:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.153169:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.153177:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.153195:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.153204:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f6b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.153299:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515769856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58881us (59130us total) trans 4294973325 rc 0/0 00000100:00100000:1.0:1789760770.153309:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6037 00000100:00100000:1.0:1789760770.153319:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748509728668 time 748559692217 state 0x2 ref 0 00000100:00100000:3.0:1789760770.163132:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.163544:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515769984 00000100:00100000:1.0:1789760770.163556:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515769984 00000100:00100000:1.0:1789760770.163609:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748509728668 time 748559692217 state 0x0 ref 1 00000100:00100000:1.0:1789760770.163625:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748559952633 time 748559692217 state 0x1 ref 1 00000100:00100000:1.0:1789760770.163656:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748559952633 time 748628944759 state 0x0 ref 1 00000100:00100000:1.0:1789760770.163668:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515769984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.163752:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24612864, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:1.0:1789760770.163761:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.164013:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760770.164018:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.164025:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.164040:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.233100:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973326 is committed 00002000:00010000:1.0:1789760770.233204:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24612864 -> 24616960 00010000:00010000:1.0:1789760770.233230:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.233243:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.233253:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.233260:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.233265:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.233274:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.233293:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.233302:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f6b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.233511:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515769984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69840us (70383us total) trans 4294973326 rc 0/0 00000100:00100000:1.0:1789760770.233526:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6038 00000100:00100000:1.0:1789760770.233535:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748559952633 time 748628944759 state 0x2 ref 0 00000100:00100000:2.0:1789760770.246058:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760770.250291:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770112 00000100:00100000:0.0:1789760770.250306:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770112 00000100:00100000:0.0:1789760770.250366:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748559952633 time 748628944759 state 0x0 ref 1 00000100:00100000:0.0:1789760770.250380:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748629205175 time 748628944759 state 0x1 ref 1 00000100:00100000:0.0:1789760770.250415:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748629205175 time 748715704455 state 0x0 ref 1 00000100:00100000:0.0:1789760770.250429:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515770112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760770.250479:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24616960, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:0.0:1789760770.250488:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.250503:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760770.250507:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.250513:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.250525:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.281174:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24616960 -> 24621056 00010000:00010000:0.0:1789760770.281194:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281207:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281217:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281224:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760770.281229:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281239:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281257:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281265:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.281390:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973327 is committed 00000100:00100000:0.0:1789760770.281539:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515770112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31107us (35484us total) trans 4294973327 rc 0/0 00000100:00100000:0.0:1789760770.281549:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6039 00000100:00100000:0.0:1789760770.281559:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748629205175 time 748715704455 state 0x2 ref 0 00000100:00100000:2.0:1789760770.286817:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760770.286950:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770240 00000100:00100000:3.0:1789760770.286960:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770240 00000100:00100000:3.0:1789760770.287013:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748629205175 time 748715704455 state 0x0 ref 1 00000100:00100000:3.0:1789760770.287029:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748715964871 time 748715704455 state 0x1 ref 1 00000100:00100000:3.0:1789760770.287061:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748715964871 time 748752350120 state 0x0 ref 1 00000100:00100000:3.0:1789760770.287073:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515770240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760770.287120:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24621056, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:3.0:1789760770.287129:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.287146:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760770.287150:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.287156:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.287169:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.330377:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24621056 -> 24625152 00010000:00010000:3.0:1789760770.330395:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.330419:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.330429:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.330435:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760770.330441:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.330450:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.330470:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.330477:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.330568:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515770240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43493us (43757us total) trans 4294973328 rc 0/0 00000100:00100000:3.0:1789760770.330578:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6040 00000100:00100000:3.0:1789760770.330588:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748715964871 time 748752350120 state 0x2 ref 0 00000100:00100000:3.0:1789760770.334100:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.334291:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770368 00000100:00100000:1.0:1789760770.334301:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770368 00000100:00100000:1.0:1789760770.334355:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748715964871 time 748752350120 state 0x0 ref 1 00000100:00100000:1.0:1789760770.334370:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748752610536 time 748752350120 state 0x1 ref 1 00000100:00100000:1.0:1789760770.334402:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748752610536 time 748799691432 state 0x0 ref 1 00000100:00100000:1.0:1789760770.334422:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515770368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.334470:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24625152, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:1.0:1789760770.334479:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.334495:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760770.334499:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.334505:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.334519:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.335363:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973328 is committed 00010000:00010000:1.0:1789760770.366095:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760770.374273:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973329 is committed 00002000:00010000:0.0:1789760770.374521:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24625152 -> 24629248 00010000:00010000:0.0:1789760770.374537:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.374550:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.374559:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.374572:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760770.374576:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.374585:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.374604:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.374611:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f6ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.374751:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515770368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40291us (40612us total) trans 4294973329 rc 0/0 00000100:00100000:0.0:1789760770.374761:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6041 00000100:00100000:0.0:1789760770.374780:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748752610536 time 748799691432 state 0x2 ref 0 00000100:00100000:1.0:1789760770.378673:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760770.378914:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770496 00000100:00100000:0.0:1789760770.378923:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770496 00000100:00100000:0.0:1789760770.378976:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748752610536 time 748799691432 state 0x0 ref 1 00000100:00100000:0.0:1789760770.378991:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748799951848 time 748799691432 state 0x1 ref 1 00000100:00100000:0.0:1789760770.379020:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748799951848 time 748844309373 state 0x0 ref 1 00000100:00100000:0.0:1789760770.379036:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515770496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760770.379084:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24629248, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:0.0:1789760770.379093:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.379109:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760770.379114:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.379119:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.379133:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.427069:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973330 is committed 00002000:00010000:0.0:1789760770.427214:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24629248 -> 24633344 00010000:00010000:0.0:1789760770.427231:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.427243:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.427252:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.427259:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760770.427263:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.427272:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.427302:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.427311:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f6d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.427417:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515770496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48369us (48741us total) trans 4294973330 rc 0/0 00000100:00100000:0.0:1789760770.427428:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6042 00000100:00100000:0.0:1789760770.427438:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748799951848 time 748844309373 state 0x2 ref 0 00000100:00100000:1.0:1789760770.431803:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760770.432038:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770624 00000100:00100000:2.0:1789760770.432047:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770624 00000100:00100000:2.0:1789760770.432101:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748799951848 time 748844309373 state 0x0 ref 1 00000100:00100000:2.0:1789760770.432115:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748844569789 time 748844309373 state 0x1 ref 1 00000100:00100000:2.0:1789760770.432150:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748844569789 time 748897439218 state 0x0 ref 1 00000100:00100000:2.0:1789760770.432164:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515770624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760770.432211:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24633344, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:2.0:1789760770.432220:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.432236:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760770.432240:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.432246:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.432259:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.466351:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24633344 -> 24637440 00010000:00010000:1.0:1789760770.466369:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.466382:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.466391:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.466399:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.466403:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.466485:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.466505:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.466514:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f6dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.466602:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515770624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34437us (34840us total) trans 4294973331 rc 0/0 00000100:00100000:1.0:1789760770.466612:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6043 00000100:00100000:1.0:1789760770.466621:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748844569789 time 748897439218 state 0x2 ref 0 00000001:00080000:3.0:1789760770.466877:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973331 is committed 00000100:00100000:1.0:1789760770.471229:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760770.471572:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770752 00000100:00100000:3.0:1789760770.471582:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770752 00000100:00100000:3.0:1789760770.471628:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748844569789 time 748897439218 state 0x0 ref 1 00000100:00100000:3.0:1789760770.471639:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748897699634 time 748897439218 state 0x1 ref 1 00000100:00100000:3.0:1789760770.471663:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748897699634 time 748936952587 state 0x0 ref 1 00000100:00100000:3.0:1789760770.471676:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515770752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760770.472028:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24637440, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:2.0:1789760770.472038:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.472053:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760770.472058:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.472064:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.472077:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.501501:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24637440 -> 24641536 00010000:00010000:1.0:1789760770.501521:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.501533:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.501543:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.501550:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.501555:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.501564:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.501583:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.501590:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f6e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.501703:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515770752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30024us (30477us total) trans 4294973332 rc 0/0 00000100:00100000:1.0:1789760770.501747:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6044 00000100:00100000:1.0:1789760770.501757:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748897699634 time 748936952587 state 0x2 ref 0 00000001:00080000:3.0:1789760770.501941:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973332 is committed 00000100:00100000:0.0:1789760770.505936:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760770.506076:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515770880 00000100:00100000:2.0:1789760770.506085:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515770880 00000100:00100000:2.0:1789760770.506217:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748897699634 time 748936952587 state 0x0 ref 1 00000100:00100000:2.0:1789760770.506233:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748937213003 time 748936952587 state 0x1 ref 1 00000100:00100000:2.0:1789760770.506264:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748937213003 time 748971553334 state 0x0 ref 1 00000100:00100000:2.0:1789760770.506278:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515770880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760770.506500:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24641536, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:2.0:1789760770.506509:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.506528:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760770.506532:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.506540:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.506554:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.544557:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973333 is committed 00002000:00010000:3.0:1789760770.544651:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24641536 -> 24645632 00010000:00010000:3.0:1789760770.544667:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.544680:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.544783:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.544791:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760770.544796:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.544805:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.544827:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.544835:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f6ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.544962:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515770880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38683us (39031us total) trans 4294973333 rc 0/0 00000100:00100000:3.0:1789760770.544977:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6045 00000100:00100000:3.0:1789760770.544987:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748937213003 time 748971553334 state 0x2 ref 0 00000100:00100000:0.0:1789760770.551071:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760770.551246:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771008 00000100:00100000:3.0:1789760770.551255:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771008 00000100:00100000:3.0:1789760770.551305:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748937213003 time 748971553334 state 0x0 ref 1 00000100:00100000:3.0:1789760770.551320:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748971813750 time 748971553334 state 0x1 ref 1 00000100:00100000:3.0:1789760770.551401:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748971813750 time 749016688990 state 0x0 ref 1 00000100:00100000:3.0:1789760770.551485:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760770.551533:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24645632, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:3.0:1789760770.551542:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.551558:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.551562:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.551568:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.551580:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.592939:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24645632 -> 24649728 00010000:00010000:1.0:1789760770.592963:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.592977:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.592986:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.592993:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.592998:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.593008:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.593028:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.593042:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1f6f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.593136:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515771008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41650us (42066us total) trans 4294973334 rc 0/0 00000100:00100000:1.0:1789760770.593146:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6046 00000100:00100000:1.0:1789760770.593156:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748971813750 time 749016688990 state 0x2 ref 0 00000001:00080000:1.0:1789760770.593192:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973334 is committed 00000100:00100000:1.0:1789760770.596477:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760770.596901:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771136 00000100:00100000:2.0:1789760770.596912:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771136 00000100:00100000:1.0:1789760770.597074:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 748971813750 time 749016688990 state 0x0 ref 1 00000100:00100000:1.0:1789760770.597092:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749016949406 time 749016688990 state 0x1 ref 1 00000100:00100000:1.0:1789760770.597233:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749016949406 time 749062521111 state 0x0 ref 1 00000100:00100000:1.0:1789760770.597247:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.597300:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24649728, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:1.0:1789760770.597310:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597327:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.597331:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.597339:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597359:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.630394:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24649728 -> 24653824 00010000:00010000:1.0:1789760770.630426:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.630440:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.630448:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.630456:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.630461:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.630471:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.630492:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.630509:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f6f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.630613:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515771136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33363us (34140us total) trans 4294973335 rc 0/0 00000100:00100000:1.0:1789760770.630625:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6047 00000100:00100000:1.0:1789760770.630634:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749016949406 time 749062521111 state 0x2 ref 0 00000001:00080000:3.0:1789760770.634826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973335 is committed 00000100:00100000:2.0:1789760770.635170:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760770.635700:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771264 00000100:00100000:0.0:1789760770.635734:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771264 00000100:00100000:0.0:1789760770.635799:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749016949406 time 749062521111 state 0x0 ref 1 00000100:00100000:0.0:1789760770.635814:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749062781527 time 749062521111 state 0x1 ref 1 00000100:00100000:0.0:1789760770.635843:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749062781527 time 749101132865 state 0x0 ref 1 00000100:00100000:0.0:1789760770.635856:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760770.635906:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24653824, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:0.0:1789760770.635915:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.635931:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760770.635935:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.635941:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.635956:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.668275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973336 is committed 00002000:00010000:1.0:1789760770.668410:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24653824 -> 24657920 00010000:00010000:1.0:1789760770.668425:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.668438:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.668448:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.668456:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.668461:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.668470:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.668490:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.668504:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f6ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.668593:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32734us (33422us total) trans 4294973336 rc 0/0 00000100:00100000:1.0:1789760770.668603:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6048 00000100:00100000:1.0:1789760770.668613:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749062781527 time 749101132865 state 0x2 ref 0 00000100:00100000:2.0:1789760770.671940:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.672211:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771392 00000100:00100000:1.0:1789760770.672221:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771392 00000100:00100000:1.0:1789760770.672275:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749062781527 time 749101132865 state 0x0 ref 1 00000100:00100000:1.0:1789760770.672366:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749101393281 time 749101132865 state 0x1 ref 1 00000100:00100000:1.0:1789760770.672400:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749101393281 time 749137689083 state 0x0 ref 1 00000100:00100000:1.0:1789760770.672414:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.672462:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24657920, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:1.0:1789760770.672471:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.672486:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.672491:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.672497:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.672516:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.717822:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973337 is committed 00002000:00010000:1.0:1789760770.717971:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24657920 -> 24662016 00010000:00010000:1.0:1789760770.717987:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.717999:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718008:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718015:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.718019:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718029:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718045:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.718053:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f706 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.718232:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45815us (46295us total) trans 4294973337 rc 0/0 00000100:00100000:1.0:1789760770.718242:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6049 00000100:00100000:1.0:1789760770.718251:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749101393281 time 749137689083 state 0x2 ref 0 00000100:00100000:1.0:1789760770.724330:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760770.724590:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771520 00000100:00100000:3.0:1789760770.724600:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771520 00000100:00100000:3.0:1789760770.724656:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749101393281 time 749137689083 state 0x0 ref 1 00000100:00100000:3.0:1789760770.724689:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749137949499 time 749137689083 state 0x1 ref 1 00000100:00100000:3.0:1789760770.724736:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749137949499 time 749190025177 state 0x0 ref 1 00000100:00100000:3.0:1789760770.724749:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760770.724950:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24662016, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:3.0:1789760770.724959:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.724974:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.724978:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.724985:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.724998:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.771034:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973338 is committed 00002000:00010000:1.0:1789760770.771199:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24662016 -> 24666112 00010000:00010000:1.0:1789760770.771215:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.771227:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.771236:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.771244:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.771249:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.771258:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.771276:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.771290:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1f70d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.771483:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46661us (47086us total) trans 4294973338 rc 0/0 00000100:00100000:1.0:1789760770.771495:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6050 00000100:00100000:1.0:1789760770.771504:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749137949499 time 749190025177 state 0x2 ref 0 00000100:00100000:2.0:1789760770.776153:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760770.776240:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771648 00000100:00100000:3.0:1789760770.776251:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771648 00000100:00100000:3.0:1789760770.776304:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749137949499 time 749190025177 state 0x0 ref 1 00000100:00100000:3.0:1789760770.776320:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749190285593 time 749190025177 state 0x1 ref 1 00000100:00100000:3.0:1789760770.776389:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749190285593 time 749241677116 state 0x0 ref 1 00000100:00100000:3.0:1789760770.776403:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515771648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760770.776460:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24666112, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:3.0:1789760770.776469:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.776483:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.776487:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.776493:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.776508:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.827288:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973339 is committed 00002000:00010000:3.0:1789760770.827396:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24666112 -> 24670208 00010000:00010000:3.0:1789760770.827501:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.827514:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.827522:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.827529:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760770.827534:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.827543:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.827563:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.827570:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f714 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.827667:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515771648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51262us (51519us total) trans 4294973339 rc 0/0 00000100:00100000:3.0:1789760770.827678:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6051 00000100:00100000:3.0:1789760770.827696:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749190285593 time 749241677116 state 0x2 ref 0 00000100:00100000:2.0:1789760770.831239:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760770.831315:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771776 00000100:00100000:3.0:1789760770.831325:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771776 00000100:00100000:3.0:1789760770.831376:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749190285593 time 749241677116 state 0x0 ref 1 00000100:00100000:3.0:1789760770.831392:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749241937532 time 749241677116 state 0x1 ref 1 00000100:00100000:3.0:1789760770.831432:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749241937532 time 749296720639 state 0x0 ref 1 00000100:00100000:3.0:1789760770.831444:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760770.831495:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24670208, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xc007 00010000:00010000:3.0:1789760770.831504:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.831520:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.831525:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.831531:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.831545:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.864150:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973340 is committed 00002000:00010000:3.0:1789760770.864267:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24670208 -> 24674304 00002000:00010000:3.0:1789760770.864272:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:3.0:1789760770.864275:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00010000:00010000:3.0:1789760770.864290:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.864303:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.864312:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.864319:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760770.864323:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.864338:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.864358:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.864365:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f71b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.864474:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515771776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33027us (33240us total) trans 4294973340 rc 0/0 00000100:00100000:3.0:1789760770.864485:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6052 00000100:00100000:3.0:1789760770.864495:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749241937532 time 749296720639 state 0x2 ref 0 00000100:00100000:1.0:1789760770.868192:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760770.868284:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515771904 00000100:00100000:0.0:1789760770.868294:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515771904 00000100:00100000:0.0:1789760770.868342:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749241937532 time 749296720639 state 0x0 ref 1 00000100:00100000:0.0:1789760770.868356:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749296981055 time 749296720639 state 0x1 ref 1 00000100:00100000:0.0:1789760770.868379:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749296981055 time 749333669122 state 0x0 ref 1 00000100:00100000:0.0:1789760770.868390:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515771904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760770.868444:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24674304, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:0.0:1789760770.868452:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.868474:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760770.868479:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.868485:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.868497:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.897907:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973341 is committed 00002000:00010000:0.0:1789760770.897996:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24674304 -> 24678400 00010000:00010000:0.0:1789760770.898011:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898023:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898031:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898038:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760770.898042:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898051:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898069:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.898077:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f722 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.898200:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515771904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29806us (30014us total) trans 4294973341 rc 0/0 00000100:00100000:0.0:1789760770.898211:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6053 00000100:00100000:0.0:1789760770.898221:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749296981055 time 749333669122 state 0x2 ref 0 00000100:00100000:0.0:1789760770.901611:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.901813:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772032 00000100:00100000:1.0:1789760770.901822:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772032 00000100:00100000:1.0:1789760770.901875:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749296981055 time 749333669122 state 0x0 ref 1 00000100:00100000:1.0:1789760770.901889:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749333929538 time 749333669122 state 0x1 ref 1 00000100:00100000:1.0:1789760770.901958:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749333929538 time 749367247136 state 0x0 ref 1 00000100:00100000:1.0:1789760770.901972:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515772032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.902021:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24678400, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760770.902031:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.902047:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.902053:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.902060:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.902074:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.925787:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24678400 -> 24682496 00010000:00010000:1.0:1789760770.925807:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.925819:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.925828:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.925836:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760770.925841:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.925851:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.925869:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.925877:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f729 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.925961:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515772032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23988us (24356us total) trans 4294973342 rc 0/0 00000100:00100000:1.0:1789760770.925972:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6054 00000100:00100000:1.0:1789760770.925981:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749333929538 time 749367247136 state 0x2 ref 0 00000001:00080000:1.0:1789760770.928484:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973342 is committed 00000100:00100000:0.0:1789760770.939590:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.939891:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772160 00000100:00100000:1.0:1789760770.939901:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772160 00000100:00100000:1.0:1789760770.939952:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749333929538 time 749367247136 state 0x0 ref 1 00000100:00100000:1.0:1789760770.939968:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749367507552 time 749367247136 state 0x1 ref 1 00000100:00100000:1.0:1789760770.939998:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749367507552 time 749405286970 state 0x0 ref 1 00000100:00100000:1.0:1789760770.940010:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515772160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.940056:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24682496, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760770.940065:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.940080:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.940084:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.940090:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.940105:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.978957:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973343 is committed 00002000:00010000:3.0:1789760770.979118:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24682496 -> 24686592 00010000:00010000:3.0:1789760770.979135:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.979148:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.979157:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.979170:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760770.979174:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.979183:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.979203:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.979210:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f730 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.979310:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515772160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39297us (39725us total) trans 4294973343 rc 0/0 00000100:00100000:3.0:1789760770.979321:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6055 00000100:00100000:3.0:1789760770.979331:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749367507552 time 749405286970 state 0x2 ref 0 00000100:00100000:2.0:1789760770.982453:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760770.982551:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772288 00000100:00100000:1.0:1789760770.982562:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772288 00000100:00100000:1.0:1789760770.982622:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749367507552 time 749405286970 state 0x0 ref 1 00000100:00100000:1.0:1789760770.982637:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749405547386 time 749405286970 state 0x1 ref 1 00000100:00100000:1.0:1789760770.982747:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749405547386 time 749448035724 state 0x0 ref 1 00000100:00100000:1.0:1789760770.982770:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515772288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760770.982834:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24686592, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760770.982843:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.982859:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.982864:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.982870:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.982884:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.029514:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24686592 -> 24690688 00010000:00010000:2.0:1789760771.029536:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.029608:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.029618:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.029626:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760771.029631:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.029641:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.029661:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.029669:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f737 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.029812:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515772288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47048us (47363us total) trans 4294973344 rc 0/0 00000100:00100000:2.0:1789760771.029824:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6056 00000100:00100000:2.0:1789760771.029833:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749405547386 time 749448035724 state 0x2 ref 0 00000001:00080000:0.0:1789760771.030058:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973344 is committed 00000100:00100000:2.0:1789760771.036638:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760771.036792:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772416 00000100:00100000:3.0:1789760771.036803:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772416 00000100:00100000:3.0:1789760771.036855:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749405547386 time 749448035724 state 0x0 ref 1 00000100:00100000:3.0:1789760771.036869:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749448296140 time 749448035724 state 0x1 ref 1 00000100:00100000:3.0:1789760771.036900:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749448296140 time 749502189130 state 0x0 ref 1 00000100:00100000:3.0:1789760771.036925:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515772416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760771.036972:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24690688, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:3.0:1789760771.036980:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.036996:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760771.037001:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.037008:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.037022:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.110577:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973345 is committed 00002000:00010000:1.0:1789760771.110704:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24690688 -> 24694784 00010000:00010000:1.0:1789760771.110745:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.110774:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.110790:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.110797:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760771.110801:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.110810:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.110838:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.110847:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1f73e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.110943:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515772416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74023us (74308us total) trans 4294973345 rc 0/0 00000100:00100000:1.0:1789760771.110952:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6057 00000100:00100000:1.0:1789760771.110962:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749448296140 time 749502189130 state 0x2 ref 0 00000100:00100000:2.0:1789760771.122135:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760771.122287:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772544 00000100:00100000:3.0:1789760771.122297:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772544 00000100:00100000:3.0:1789760771.122350:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749448296140 time 749502189130 state 0x0 ref 1 00000100:00100000:3.0:1789760771.122367:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749502449546 time 749502189130 state 0x1 ref 1 00000100:00100000:3.0:1789760771.122403:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749502449546 time 749587691461 state 0x0 ref 1 00000100:00100000:3.0:1789760771.122416:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515772544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760771.122466:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24694784, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:3.0:1789760771.122475:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.122495:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760771.122500:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.122506:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.122521:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.173035:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24694784 -> 24698880 00010000:00010000:1.0:1789760771.173064:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.173076:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.173085:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.173091:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760771.173096:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.173105:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.173122:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.173129:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1f745 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.173218:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515772544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50800us (51089us total) trans 4294973346 rc 0/0 00000100:00100000:1.0:1789760771.173229:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6058 00000100:00100000:1.0:1789760771.173239:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749502449546 time 749587691461 state 0x2 ref 0 00000001:00080000:3.0:1789760771.177260:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973346 is committed 00000100:00100000:1.0:1789760771.177364:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760771.177523:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772672 00000100:00100000:0.0:1789760771.177532:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772672 00000100:00100000:0.0:1789760771.177586:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749502449546 time 749587691461 state 0x0 ref 1 00000100:00100000:0.0:1789760771.177602:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749587951877 time 749587691461 state 0x1 ref 1 00000100:00100000:0.0:1789760771.177674:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749587951877 time 749642961899 state 0x0 ref 1 00000100:00100000:0.0:1789760771.177698:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515772672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760771.177930:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24698880, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:0.0:1789760771.177941:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.177963:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760771.177967:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.177975:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.177988:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.214539:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24698880 -> 24702976 00010000:00010000:1.0:1789760771.214560:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.214573:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.214582:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.214589:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760771.214594:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.214603:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.214623:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.214630:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f74c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.214787:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515772672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37064us (37399us total) trans 4294973347 rc 0/0 00000100:00100000:1.0:1789760771.214809:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6059 00000100:00100000:1.0:1789760771.214819:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749587951877 time 749642961899 state 0x2 ref 0 00000001:00080000:3.0:1789760771.215934:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973347 is committed 00000100:00100000:2.0:1789760771.219256:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.219517:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772800 00000100:00100000:1.0:1789760771.219527:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772800 00000100:00100000:1.0:1789760771.219580:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749587951877 time 749642961899 state 0x0 ref 1 00000100:00100000:1.0:1789760771.219596:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749643222315 time 749642961899 state 0x1 ref 1 00000100:00100000:1.0:1789760771.219660:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749643222315 time 749684948851 state 0x0 ref 1 00000100:00100000:1.0:1789760771.219673:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515772800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.219781:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24702976, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.219805:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.219822:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760771.219827:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.219833:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.219849:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.257903:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24702976 -> 24707072 00010000:00010000:1.0:1789760771.257924:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257938:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257946:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257953:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760771.257958:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257966:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257985:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.257992:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f753 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.258134:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515772800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38458us (38880us total) trans 4294973348 rc 0/0 00000100:00100000:1.0:1789760771.258146:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6060 00000100:00100000:1.0:1789760771.258165:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749643222315 time 749684948851 state 0x2 ref 0 00000001:00080000:3.0:1789760771.258432:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973348 is committed 00000100:00100000:2.0:1789760771.268134:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.268297:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515772928 00000100:00100000:1.0:1789760771.268307:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515772928 00000100:00100000:1.0:1789760771.268350:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749643222315 time 749684948851 state 0x0 ref 1 00000100:00100000:1.0:1789760771.268362:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749685209267 time 749684948851 state 0x1 ref 1 00000100:00100000:1.0:1789760771.268389:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749685209267 time 749733680016 state 0x0 ref 1 00000100:00100000:1.0:1789760771.268398:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515772928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.268440:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24707072, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.268449:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.268465:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760771.268469:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.268475:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.268495:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.315044:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973349 is committed 00002000:00010000:3.0:1789760771.315148:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24707072 -> 24711168 00010000:00010000:3.0:1789760771.315177:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.315191:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.315201:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.315208:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760771.315213:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.315234:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.315255:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.315264:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f75a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.315567:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515772928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47163us (47437us total) trans 4294973349 rc 0/0 00000100:00100000:3.0:1789760771.315578:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6061 00000100:00100000:3.0:1789760771.315589:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749685209267 time 749733680016 state 0x2 ref 0 00000100:00100000:0.0:1789760771.320860:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.325299:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773056 00000100:00100000:1.0:1789760771.325314:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773056 00000100:00100000:1.0:1789760771.325377:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749685209267 time 749733680016 state 0x0 ref 1 00000100:00100000:1.0:1789760771.325396:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749733940432 time 749733680016 state 0x1 ref 1 00000100:00100000:1.0:1789760771.325421:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749733940432 time 749790710840 state 0x0 ref 1 00000100:00100000:1.0:1789760771.325438:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515773056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.325484:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24711168, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.325493:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.325510:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760771.325514:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.325520:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.325534:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.391626:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760771.392204:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24711168 -> 24715264 00010000:00010000:2.0:1789760771.392221:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392234:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392242:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392250:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760771.392254:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392263:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392281:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.392290:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f761 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.392387:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515773056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66947us (71532us total) trans 4294973350 rc 0/0 00000100:00100000:2.0:1789760771.392399:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6062 00000100:00100000:2.0:1789760771.392419:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749733940432 time 749790710840 state 0x2 ref 0 00000001:00080000:3.0:1789760771.392582:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973350 is committed 00000100:00100000:0.0:1789760771.400908:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760771.401005:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773184 00000100:00100000:3.0:1789760771.401017:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773184 00000100:00100000:3.0:1789760771.401084:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749733940432 time 749790710840 state 0x0 ref 1 00000100:00100000:3.0:1789760771.401099:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749790971256 time 749790710840 state 0x1 ref 1 00000100:00100000:3.0:1789760771.401129:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749790971256 time 749866418436 state 0x0 ref 1 00000100:00100000:3.0:1789760771.401143:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515773184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760771.401194:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24715264, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:3.0:1789760771.401204:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.401226:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760771.401231:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.401238:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.401251:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.465190:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973351 is committed 00002000:00010000:3.0:1789760771.465462:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24715264 -> 24719360 00010000:00010000:3.0:1789760771.465477:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.465491:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.465500:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.465507:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760771.465512:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.465521:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.465540:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.465549:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f768 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.465637:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515773184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64492us (64734us total) trans 4294973351 rc 0/0 00000100:00100000:3.0:1789760771.465649:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6063 00000100:00100000:3.0:1789760771.465659:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749790971256 time 749866418436 state 0x2 ref 0 00000100:00100000:1.0:1789760771.472511:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760771.472616:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773312 00000100:00100000:3.0:1789760771.472626:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773312 00000100:00100000:3.0:1789760771.472679:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749790971256 time 749866418436 state 0x0 ref 1 00000100:00100000:3.0:1789760771.472696:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749866678852 time 749866418436 state 0x1 ref 1 00000100:00100000:1.0:1789760771.472976:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749866678852 time 749938262884 state 0x0 ref 1 00000100:00100000:1.0:1789760771.472991:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515773312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.473046:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24719360, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.473054:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.473070:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760771.473075:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.473082:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.473102:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.501739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973352 is committed 00002000:00010000:0.0:1789760771.501850:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24719360 -> 24723456 00010000:00010000:0.0:1789760771.501866:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.501879:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.501889:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.501896:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760771.501901:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.501910:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.501929:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.501938:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f76f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.502025:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515773312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29033us (29525us total) trans 4294973352 rc 0/0 00000100:00100000:0.0:1789760771.502035:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6064 00000100:00100000:0.0:1789760771.502044:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749866678852 time 749938262884 state 0x2 ref 0 00000100:00100000:2.0:1789760771.506882:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.506974:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773440 00000100:00100000:1.0:1789760771.506985:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773440 00000100:00100000:1.0:1789760771.507039:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749866678852 time 749938262884 state 0x0 ref 1 00000100:00100000:1.0:1789760771.507054:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749938523300 time 749938262884 state 0x1 ref 1 00000100:00100000:1.0:1789760771.507085:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749938523300 time 749972374345 state 0x0 ref 1 00000100:00100000:1.0:1789760771.507098:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515773440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.507147:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24723456, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.507157:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.507174:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760771.507179:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.507186:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.507200:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.545485:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973353 is committed 00002000:00010000:0.0:1789760771.545675:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24723456 -> 24727552 00010000:00010000:0.0:1789760771.545828:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.545840:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.545849:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.545856:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760771.545861:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.545874:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.545893:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.545900:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f776 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.546371:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515773440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39268us (39491us total) trans 4294973353 rc 0/0 00000100:00100000:1.0:1789760771.546383:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6065 00000100:00100000:1.0:1789760771.546393:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749938523300 time 749972374345 state 0x2 ref 0 00000100:00100000:3.0:1789760771.550959:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.551090:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773568 00000100:00100000:1.0:1789760771.551100:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773568 00000100:00100000:1.0:1789760771.551151:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749938523300 time 749972374345 state 0x0 ref 1 00000100:00100000:1.0:1789760771.551165:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749972634761 time 749972374345 state 0x1 ref 1 00000100:00100000:1.0:1789760771.551237:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749972634761 time 750016526357 state 0x0 ref 1 00000100:00100000:1.0:1789760771.551250:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515773568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.551298:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24727552, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.551306:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.551320:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.551324:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.551330:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.551343:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.590754:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24727552 -> 24731648 00010000:00010000:2.0:1789760771.590781:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.590794:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.590804:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.590811:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760771.590816:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.590825:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.590844:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.590853:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f77d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.590937:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515773568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39683us (39982us total) trans 4294973354 rc 0/0 00000100:00100000:2.0:1789760771.590947:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6066 00000100:00100000:2.0:1789760771.590958:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749972634761 time 750016526357 state 0x2 ref 0 00000001:00080000:0.0:1789760771.591194:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973354 is committed 00000100:00100000:2.0:1789760771.595345:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.595454:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773696 00000100:00100000:1.0:1789760771.595464:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773696 00000100:00100000:1.0:1789760771.595519:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 749972634761 time 750016526357 state 0x0 ref 1 00000100:00100000:1.0:1789760771.595533:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750016786773 time 750016526357 state 0x1 ref 1 00000100:00100000:1.0:1789760771.595615:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750016786773 time 750060903183 state 0x0 ref 1 00000100:00100000:1.0:1789760771.595627:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caa450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515773696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.595678:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24731648, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.595686:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.595702:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.595746:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.595754:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.595778:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.649658:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973355 is committed 00002000:00010000:3.0:1789760771.649973:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24731648 -> 24735744 00010000:00010000:3.0:1789760771.649989:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.650002:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.650011:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.650023:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760771.650028:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.650037:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.650131:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.650140:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f784 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.650235:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caa450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515773696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54605us (54970us total) trans 4294973355 rc 0/0 00000100:00100000:3.0:1789760771.650246:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6067 00000100:00100000:3.0:1789760771.650255:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750016786773 time 750060903183 state 0x2 ref 0 00000100:00100000:3.0:1789760771.654151:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760771.654254:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773824 00000100:00100000:2.0:1789760771.654265:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773824 00000100:00100000:2.0:1789760771.654321:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750016786773 time 750060903183 state 0x0 ref 1 00000100:00100000:2.0:1789760771.654338:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750061163599 time 750060903183 state 0x1 ref 1 00000100:00100000:2.0:1789760771.654371:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750061163599 time 750119659700 state 0x0 ref 1 00000100:00100000:2.0:1789760771.654384:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515773824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760771.654438:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24735744, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:2.0:1789760771.654446:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.654467:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.654471:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.654478:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.654494:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.691605:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973356 is committed 00002000:00010000:2.0:1789760771.691848:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24735744 -> 24739840 00010000:00010000:2.0:1789760771.691864:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.691875:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.691884:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.691891:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760771.691896:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.691906:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.691929:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.691937:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f78b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.692031:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515773824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37644us (37884us total) trans 4294973356 rc 0/0 00000100:00100000:2.0:1789760771.692041:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6068 00000100:00100000:2.0:1789760771.692565:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750061163599 time 750119659700 state 0x2 ref 0 00000100:00100000:2.0:1789760771.698926:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760771.699107:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515773952 00000100:00100000:3.0:1789760771.699117:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515773952 00000100:00100000:3.0:1789760771.699175:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750061163599 time 750119659700 state 0x0 ref 1 00000100:00100000:3.0:1789760771.699191:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750119920116 time 750119659700 state 0x1 ref 1 00000100:00100000:3.0:1789760771.699218:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750119920116 time 750164508398 state 0x0 ref 1 00000100:00100000:3.0:1789760771.699230:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515773952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760771.699275:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24739840, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:3.0:1789760771.699283:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.699297:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760771.699301:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.699306:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.699328:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.767557:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973357 is committed 00002000:00010000:3.0:1789760771.768626:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24739840 -> 24743936 00010000:00010000:3.0:1789760771.768643:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.768657:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.768666:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.768673:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760771.768679:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.768688:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.769192:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.769203:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f792 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.769393:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515773952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70159us (70472us total) trans 4294973357 rc 0/0 00000100:00100000:3.0:1789760771.769404:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6069 00000100:00100000:3.0:1789760771.769488:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750119920116 time 750164508398 state 0x2 ref 0 00000100:00100000:1.0:1789760771.778800:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760771.778845:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774080 00000100:00100000:0.0:1789760771.778857:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774080 00000100:00100000:0.0:1789760771.778912:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750119920116 time 750164508398 state 0x0 ref 1 00000100:00100000:0.0:1789760771.778928:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750164768814 time 750164508398 state 0x1 ref 1 00000100:00100000:0.0:1789760771.778969:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750164768814 time 750244257709 state 0x0 ref 1 00000100:00100000:0.0:1789760771.778982:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515774080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760771.779028:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24743936, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:0.0:1789760771.779036:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.779062:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.779068:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.779075:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.779089:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.828526:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973358 is committed 00002000:00010000:0.0:1789760771.828868:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24743936 -> 24748032 00010000:00010000:0.0:1789760771.828883:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828895:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828903:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828909:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760771.828914:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828922:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828941:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828949:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1f799 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.829040:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515774080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50055us (50245us total) trans 4294973358 rc 0/0 00000100:00100000:0.0:1789760771.829050:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6070 00000100:00100000:0.0:1789760771.829059:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750164768814 time 750244257709 state 0x2 ref 0 00000100:00100000:2.0:1789760771.834339:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.834425:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774208 00000100:00100000:1.0:1789760771.834435:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774208 00000100:00100000:1.0:1789760771.834486:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750164768814 time 750244257709 state 0x0 ref 1 00000100:00100000:1.0:1789760771.834501:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750244518125 time 750244257709 state 0x1 ref 1 00000100:00100000:1.0:1789760771.834539:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750244518125 time 750299827937 state 0x0 ref 1 00000100:00100000:1.0:1789760771.834551:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515774208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.834596:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24748032, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xc007 00010000:00010000:1.0:1789760771.834605:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.834619:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.834623:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.834629:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.834642:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.903316:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973359 is committed 00002000:00010000:2.0:1789760771.910987:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24748032 -> 24752128 00002000:00010000:2.0:1789760771.910998:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760771 -> 1789760772 00002000:00010000:2.0:1789760771.911000:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760771 -> 1789760772 00010000:00010000:2.0:1789760771.911013:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.911028:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.911038:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.911046:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760771.911051:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.911061:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.911081:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.911089:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1f7a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.911180:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515774208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 76626us (76845us total) trans 4294973359 rc 0/0 00000100:00100000:2.0:1789760771.911191:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6071 00000100:00100000:2.0:1789760771.911200:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750244518125 time 750299827937 state 0x2 ref 0 00000100:00100000:2.0:1789760771.914304:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.914393:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774336 00000100:00100000:1.0:1789760771.914402:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774336 00000100:00100000:1.0:1789760771.914454:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750244518125 time 750299827937 state 0x0 ref 1 00000100:00100000:1.0:1789760771.914468:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750300088353 time 750299827937 state 0x1 ref 1 00000100:00100000:1.0:1789760771.914498:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750300088353 time 750379787081 state 0x0 ref 1 00000100:00100000:1.0:1789760771.914510:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760771.914557:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24752128, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760771.914566:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.914581:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.914586:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.914592:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.914606:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.943027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973360 is committed 00002000:00010000:1.0:1789760771.943139:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24752128 -> 24756224 00010000:00010000:1.0:1789760771.943154:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.943166:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.943176:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.943184:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760771.943188:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.943198:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.943231:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.943248:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.943337:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28826us (29039us total) trans 4294973360 rc 0/0 00000100:00100000:1.0:1789760771.943347:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6072 00000100:00100000:1.0:1789760771.943357:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750300088353 time 750379787081 state 0x2 ref 0 00000100:00100000:0.0:1789760771.952154:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760771.952263:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774464 00000100:00100000:3.0:1789760771.952273:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774464 00000100:00100000:3.0:1789760771.952325:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750300088353 time 750379787081 state 0x0 ref 1 00000100:00100000:3.0:1789760771.952342:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750380047497 time 750379787081 state 0x1 ref 1 00000100:00100000:3.0:1789760771.952373:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750380047497 time 750417662748 state 0x0 ref 1 00000100:00100000:3.0:1789760771.952385:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760771.952434:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24756224, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:3.0:1789760771.952444:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.952460:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760771.952464:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.952470:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.952484:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.985900:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973361 is committed 00002000:00010000:1.0:1789760771.986062:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24756224 -> 24760320 00010000:00010000:1.0:1789760771.986079:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.986093:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.986101:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.986108:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760771.986112:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.986125:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.986144:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.986152:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1f7ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.986240:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33852us (34091us total) trans 4294973361 rc 0/0 00000100:00100000:1.0:1789760771.986250:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6073 00000100:00100000:1.0:1789760771.986260:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750380047497 time 750417662748 state 0x2 ref 0 00000100:00100000:2.0:1789760771.999513:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760771.999637:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774592 00000100:00100000:1.0:1789760771.999647:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774592 00000100:00100000:1.0:1789760772.000983:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750380047497 time 750417662748 state 0x0 ref 1 00000100:00100000:1.0:1789760772.001008:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750417923164 time 750417662748 state 0x1 ref 1 00000100:00100000:1.0:1789760772.001107:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750417923164 time 750466396534 state 0x0 ref 1 00000100:00100000:1.0:1789760772.001141:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515774592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.001191:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24760320, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760772.001200:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.001216:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760772.001221:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.001228:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.001240:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760772.001885:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760772.037985:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973362 is committed 00002000:00010000:2.0:1789760772.038179:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24760320 -> 24764416 00010000:00010000:2.0:1789760772.038195:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.038208:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.038218:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.038227:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.038232:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.038242:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.038261:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.038270:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f7b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.038502:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515774592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37378us (38992us total) trans 4294973362 rc 0/0 00000100:00100000:2.0:1789760772.038513:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6074 00000100:00100000:2.0:1789760772.038523:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750417923164 time 750466396534 state 0x2 ref 0 00000100:00100000:2.0:1789760772.041318:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760772.041464:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774720 00000100:00100000:3.0:1789760772.041473:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774720 00000100:00100000:3.0:1789760772.041525:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750417923164 time 750466396534 state 0x0 ref 1 00000100:00100000:3.0:1789760772.041541:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750466656950 time 750466396534 state 0x1 ref 1 00000100:00100000:3.0:1789760772.041616:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750466656950 time 750506903860 state 0x0 ref 1 00000100:00100000:3.0:1789760772.041630:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760772.041686:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24764416, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:3.0:1789760772.041696:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.041752:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760772.041767:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.041775:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.041790:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.084773:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24764416 -> 24768512 00010000:00010000:1.0:1789760772.084801:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.084815:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.084823:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.084830:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.084835:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.084844:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.084863:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.084871:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1f7bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.084971:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515774720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43338us (43657us total) trans 4294973363 rc 0/0 00000100:00100000:1.0:1789760772.084981:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6075 00000100:00100000:1.0:1789760772.084990:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750466656950 time 750506903860 state 0x2 ref 0 00000001:00080000:0.0:1789760772.085199:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973363 is committed 00000100:00100000:0.0:1789760772.091109:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.091189:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774848 00000100:00100000:2.0:1789760772.091197:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774848 00000100:00100000:2.0:1789760772.091296:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750466656950 time 750506903860 state 0x0 ref 1 00000100:00100000:2.0:1789760772.091307:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750507164276 time 750506903860 state 0x1 ref 1 00000100:00100000:2.0:1789760772.091329:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750507164276 time 750556618719 state 0x0 ref 1 00000100:00100000:2.0:1789760772.091342:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.091568:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24768512, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.091578:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.091595:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760772.091599:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.091605:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.091620:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.131884:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973364 is committed 00002000:00010000:0.0:1789760772.132004:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24768512 -> 24772608 00010000:00010000:0.0:1789760772.132020:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.132032:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.132039:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.132045:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760772.132049:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.132058:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.132083:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.132091:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f7c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.132186:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515774848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40841us (41081us total) trans 4294973364 rc 0/0 00000100:00100000:0.0:1789760772.132195:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6076 00000100:00100000:0.0:1789760772.132204:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750507164276 time 750556618719 state 0x2 ref 0 00000100:00100000:2.0:1789760772.148528:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.149393:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515774976 00000100:00100000:1.0:1789760772.149406:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515774976 00000100:00100000:1.0:1789760772.149461:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750507164276 time 750556618719 state 0x0 ref 1 00000100:00100000:1.0:1789760772.149477:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750556879135 time 750556618719 state 0x1 ref 1 00000100:00100000:1.0:1789760772.149560:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750556879135 time 750614848534 state 0x0 ref 1 00000100:00100000:1.0:1789760772.149575:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515774976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.149623:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24772608, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760772.149632:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.149648:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760772.149653:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.149659:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.149871:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.183038:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24772608 -> 24776704 00010000:00010000:1.0:1789760772.183059:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.183072:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.183081:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.183088:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.183093:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.183102:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.183122:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.183130:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.183225:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515774976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33648us (34702us total) trans 4294973365 rc 0/0 00000100:00100000:1.0:1789760772.183235:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6077 00000100:00100000:1.0:1789760772.183245:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750556879135 time 750614848534 state 0x2 ref 0 00000001:00080000:3.0:1789760772.183316:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973365 is committed 00000100:00100000:2.0:1789760772.189229:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.189444:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775104 00000100:00100000:1.0:1789760772.189454:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775104 00000100:00100000:1.0:1789760772.189512:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750556879135 time 750614848534 state 0x0 ref 1 00000100:00100000:1.0:1789760772.189526:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750615108950 time 750614848534 state 0x1 ref 1 00000100:00100000:1.0:1789760772.189594:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750615108950 time 750654882607 state 0x0 ref 1 00000100:00100000:1.0:1789760772.189606:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515775104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.189651:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24776704, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760772.189660:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.189676:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760772.189681:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.189686:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.189701:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.246166:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24776704 -> 24780800 00010000:00010000:2.0:1789760772.246187:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.246200:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.246209:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.246216:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.246221:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.246229:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.246247:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.246254:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f7d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.246351:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515775104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56741us (57127us total) trans 4294973366 rc 0/0 00000100:00100000:2.0:1789760772.246362:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6078 00000100:00100000:2.0:1789760772.246372:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750615108950 time 750654882607 state 0x2 ref 0 00000001:00080000:3.0:1789760772.247289:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973366 is committed 00000100:00100000:2.0:1789760772.256247:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.256336:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775232 00000100:00100000:1.0:1789760772.256355:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775232 00000100:00100000:1.0:1789760772.256424:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750615108950 time 750654882607 state 0x0 ref 1 00000100:00100000:1.0:1789760772.256439:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750655143023 time 750654882607 state 0x1 ref 1 00000100:00100000:1.0:1789760772.256468:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750655143023 time 750721757663 state 0x0 ref 1 00000100:00100000:1.0:1789760772.256490:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515775232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.256537:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24780800, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760772.256546:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.256561:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760772.256565:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.256572:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.256586:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.296149:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973367 is committed 00002000:00010000:2.0:1789760772.296281:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24780800 -> 24784896 00010000:00010000:2.0:1789760772.296298:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.296310:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.296320:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.296332:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.296337:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.296351:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.296373:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.296381:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f7d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.296474:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515775232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39991us (40231us total) trans 4294973367 rc 0/0 00000100:00100000:2.0:1789760772.296486:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6079 00000100:00100000:2.0:1789760772.296495:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750655143023 time 750721757663 state 0x2 ref 0 00000100:00100000:1.0:1789760772.302666:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.304070:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775360 00000100:00100000:2.0:1789760772.304083:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775360 00000100:00100000:2.0:1789760772.304149:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750655143023 time 750721757663 state 0x0 ref 1 00000100:00100000:2.0:1789760772.304164:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750722018079 time 750721757663 state 0x1 ref 1 00000100:00100000:2.0:1789760772.304193:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750722018079 time 750769482852 state 0x0 ref 1 00000100:00100000:2.0:1789760772.304210:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515775360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.304257:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24784896, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.304266:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.304281:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760772.304286:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.304292:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.304305:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.354588:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973368 is committed 00002000:00010000:3.0:1789760772.354990:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24784896 -> 24788992 00010000:00010000:3.0:1789760772.355007:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.355019:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.355027:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.355034:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760772.355038:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.355119:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.355139:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.355147:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f7df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.355274:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515775360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51062us (52615us total) trans 4294973368 rc 0/0 00000100:00100000:3.0:1789760772.355285:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6080 00000100:00100000:3.0:1789760772.355295:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750722018079 time 750769482852 state 0x2 ref 0 00000100:00100000:3.0:1789760772.361986:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.362199:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775488 00000100:00100000:1.0:1789760772.362209:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775488 00000100:00100000:1.0:1789760772.362264:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750722018079 time 750769482852 state 0x0 ref 1 00000100:00100000:1.0:1789760772.362280:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750769743268 time 750769482852 state 0x1 ref 1 00000100:00100000:1.0:1789760772.362313:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750769743268 time 750827601787 state 0x0 ref 1 00000100:00100000:1.0:1789760772.362325:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515775488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.362375:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24788992, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760772.362384:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.362409:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760772.362414:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.362421:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.362434:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.411249:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973369 is committed 00002000:00010000:0.0:1789760772.411376:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24788992 -> 24793088 00010000:00010000:0.0:1789760772.411392:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.411478:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.411487:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.411494:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760772.411498:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.411513:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.411532:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.411541:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f7e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.411669:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515775488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49339us (49686us total) trans 4294973369 rc 0/0 00000100:00100000:0.0:1789760772.411686:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6081 00000100:00100000:0.0:1789760772.411697:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750769743268 time 750827601787 state 0x2 ref 0 00010000:00010000:0.0:1789760772.414146:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760772.417701:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760772.417939:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775616 00000100:00100000:0.0:1789760772.417949:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775616 00000100:00100000:0.0:1789760772.418012:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750769743268 time 750827601787 state 0x0 ref 1 00000100:00100000:0.0:1789760772.418027:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750827862203 time 750827601787 state 0x1 ref 1 00000100:00100000:2.0:1789760772.418251:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750827862203 time 750883537540 state 0x0 ref 1 00000100:00100000:2.0:1789760772.418266:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515775616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.418313:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24793088, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.418322:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.418336:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760772.418341:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.418347:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.418367:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.453318:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973370 is committed 00002000:00010000:2.0:1789760772.453402:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24793088 -> 24797184 00010000:00010000:2.0:1789760772.453419:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.453431:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.453440:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.453447:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.453452:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.453462:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.453834:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.453844:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.453945:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515775616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35677us (36248us total) trans 4294973370 rc 0/0 00000100:00100000:2.0:1789760772.453955:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6082 00000100:00100000:2.0:1789760772.453964:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750827862203 time 750883537540 state 0x2 ref 0 00000100:00100000:1.0:1789760772.465955:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.466190:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775744 00000100:00100000:2.0:1789760772.466200:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775744 00000100:00100000:2.0:1789760772.466259:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750827862203 time 750883537540 state 0x0 ref 1 00000100:00100000:2.0:1789760772.466274:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750883797956 time 750883537540 state 0x1 ref 1 00000100:00100000:2.0:1789760772.466306:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750883797956 time 750931595808 state 0x0 ref 1 00000100:00100000:2.0:1789760772.466318:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515775744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.466367:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24797184, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.466376:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.466391:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760772.466395:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.466402:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.466415:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.505441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973371 is committed 00002000:00010000:1.0:1789760772.505867:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24797184 -> 24801280 00010000:00010000:1.0:1789760772.505884:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.505896:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.505906:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.505913:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.505918:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.505927:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.505948:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.505956:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1f7f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.506139:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515775744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39817us (40190us total) trans 4294973371 rc 0/0 00000100:00100000:1.0:1789760772.506149:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6083 00000100:00100000:1.0:1789760772.506159:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750883797956 time 750931595808 state 0x2 ref 0 00000100:00100000:3.0:1789760772.511960:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.512154:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515775872 00000100:00100000:2.0:1789760772.512164:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515775872 00000100:00100000:2.0:1789760772.512226:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750883797956 time 750931595808 state 0x0 ref 1 00000100:00100000:2.0:1789760772.512241:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750931856224 time 750931595808 state 0x1 ref 1 00000100:00100000:2.0:1789760772.512315:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750931856224 time 750977603629 state 0x0 ref 1 00000100:00100000:2.0:1789760772.512333:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515775872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.512380:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24801280, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.512389:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.512404:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760772.512410:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.512416:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.512429:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.554244:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973372 is committed 00002000:00010000:2.0:1789760772.554358:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24801280 -> 24805376 00010000:00010000:2.0:1789760772.554373:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.554401:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.554410:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.554418:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.554422:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.554431:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.554452:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.554460:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f7fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.554551:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515775872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42219us (42594us total) trans 4294973372 rc 0/0 00000100:00100000:2.0:1789760772.554561:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6084 00000100:00100000:2.0:1789760772.554571:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750931856224 time 750977603629 state 0x2 ref 0 00000100:00100000:2.0:1789760772.559094:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760772.559318:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776000 00000100:00100000:0.0:1789760772.559328:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776000 00000100:00100000:0.0:1789760772.559381:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750931856224 time 750977603629 state 0x0 ref 1 00000100:00100000:0.0:1789760772.559398:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750977864045 time 750977603629 state 0x1 ref 1 00000100:00100000:0.0:1789760772.559428:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750977864045 time 751024717424 state 0x0 ref 1 00000100:00100000:0.0:1789760772.559442:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760772.559492:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24805376, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:0.0:1789760772.559501:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.559519:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.559524:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.559531:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.559545:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.589558:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973373 is committed 00002000:00010000:0.0:1789760772.589826:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24805376 -> 24809472 00010000:00010000:0.0:1789760772.589844:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.589857:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.589867:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.589875:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760772.589881:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.589891:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.589911:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.589919:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f802 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.590195:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30751us (31106us total) trans 4294973373 rc 0/0 00000100:00100000:3.0:1789760772.590207:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6085 00000100:00100000:3.0:1789760772.590217:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750977864045 time 751024717424 state 0x2 ref 0 00000100:00100000:1.0:1789760772.596929:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760772.605558:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776128 00000100:00100000:3.0:1789760772.605573:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776128 00000100:00100000:3.0:1789760772.605638:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 750977864045 time 751024717424 state 0x0 ref 1 00000100:00100000:3.0:1789760772.605658:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751024977840 time 751024717424 state 0x1 ref 1 00000100:00100000:3.0:1789760772.605680:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751024977840 time 751070969705 state 0x0 ref 1 00000100:00100000:3.0:1789760772.605693:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515776128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760772.605774:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24809472, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:3.0:1789760772.605784:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.605800:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.605805:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.605810:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.605824:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.607821:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8063b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544418176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.607892:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760772.607948:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544418688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.607976:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760772.608006:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.608012:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8063b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544418304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.608040:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760772.608072:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418176 00000100:00100000:3.0:1789760772.608080:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418176 00000100:00100000:3.0:1789760772.608119:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418304 00000100:00100000:3.0:1789760772.608122:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418304 00000100:00100000:3.0:1789760772.608152:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1789760772.608161:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544418176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.608211:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805634a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544418176:12345-0@lo:400:kworker.0 Request processed in 51us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.608218:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:3.0:1789760772.608234:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789760772.608241:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544418304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.608269:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544418304:12345-0@lo:400:kworker.0 Request processed in 29us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.608275:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789760772.608291:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418688 00000100:00100000:3.0:1789760772.608294:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418688 00000100:00100000:3.0:1789760772.608320:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:3.0:1789760772.608326:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544418688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.608356:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544418688:12345-0@lo:400:kworker.0 Request processed in 30us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.608362:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00080000:3.0:1789760772.608381:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760772.608385:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760772.608515:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8063b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544418048:0@lo:400:kworker.0 00000100:00100000:1.0:1789760772.608550:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760772.608576:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544418560:0@lo:400:kworker.0 00000100:00100000:1.0:1789760772.608597:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760772.608614:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760772.608619:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80620df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544418432:0@lo:400:kworker.0 00000100:00100000:1.0:1789760772.608640:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760772.608665:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418048 00000100:00100000:1.0:1789760772.608669:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418048 00000100:00100000:1.0:1789760772.608696:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:1.0:1789760772.608703:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544418048:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760772.608769:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544418048:12345-0@lo:400:kworker.0 Request processed in 66us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760772.608776:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:1.0:1789760772.608815:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8063b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544418048:0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.608894:0:5854:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418560 00000100:00100000:2.0:1789760772.608899:0:5854:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418560 00000100:00100000:2.0:1789760772.608976:0:5854:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:2.0:1789760772.608983:0:5854:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544418560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.609021:0:5854:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544418560:12345-0@lo:400:kworker.0 Request processed in 40us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760772.609028:0:5854:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:2.0:1789760772.609055:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544418560:0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.609086:0:6707:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418432 00000100:00100000:2.0:1789760772.609089:0:6707:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418432 00000100:00100000:3.0:1789760772.609145:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8063b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544418304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.609157:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8063b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544418176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760772.609169:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544418688:0@lo:400:kworker.0 00000100:00080000:1.0:1789760772.616106:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760772.616130:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760772.616529:0:6707:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1789760772.616543:0:6707:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544418432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760772.616646:0:6707:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544418432:12345-0@lo:400:kworker.0 Request processed in 107us (8004us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760772.616654:0:6707:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1789760772.617152:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80620df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544418432:0@lo:400:kworker.0 00002000:00010000:1.0:1789760772.664965:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24809472 -> 24813568 00010000:00010000:1.0:1789760772.664986:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665000:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665009:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665016:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.665022:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665030:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665050:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.665057:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1f809 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.665152:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515776128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59455us (68227us total) trans 4294973374 rc 0/0 00000100:00100000:1.0:1789760772.665163:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6086 00000100:00100000:1.0:1789760772.665172:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751024977840 time 751070969705 state 0x2 ref 0 00000001:00080000:2.0:1789760772.667590:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973374 is committed 00000100:00100000:2.0:1789760772.668427:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760772.668561:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776256 00000100:00100000:0.0:1789760772.668571:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776256 00000100:00100000:0.0:1789760772.668620:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751024977840 time 751070969705 state 0x0 ref 1 00000100:00100000:0.0:1789760772.668635:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751071230121 time 751070969705 state 0x1 ref 1 00000100:00100000:0.0:1789760772.668664:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751071230121 time 751133953159 state 0x0 ref 1 00000100:00100000:0.0:1789760772.668683:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760772.668859:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24813568, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:0.0:1789760772.668868:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668882:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.668886:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.668893:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668907:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.695699:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973375 is committed 00002000:00010000:2.0:1789760772.695908:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24813568 -> 24817664 00010000:00010000:2.0:1789760772.695924:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.695937:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.695945:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.695958:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.695963:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.695973:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.695992:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.695999:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f810 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.696127:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27447us (27701us total) trans 4294973375 rc 0/0 00000100:00100000:2.0:1789760772.696138:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6087 00000100:00100000:2.0:1789760772.696147:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751071230121 time 751133953159 state 0x2 ref 0 00000100:00100000:2.0:1789760772.700229:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.700331:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776384 00000100:00100000:1.0:1789760772.700342:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776384 00000100:00100000:1.0:1789760772.700457:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751071230121 time 751133953159 state 0x0 ref 1 00000100:00100000:1.0:1789760772.700475:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751134213575 time 751133953159 state 0x1 ref 1 00000100:00100000:1.0:1789760772.700521:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751134213575 time 751165809915 state 0x0 ref 1 00000100:00100000:1.0:1789760772.700534:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.701012:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24817664, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:1.0:1789760772.701023:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.701038:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.701042:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.701048:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.701063:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.731172:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973376 is committed 00002000:00010000:1.0:1789760772.731251:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24817664 -> 24821760 00010000:00010000:1.0:1789760772.731266:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731278:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731288:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731296:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.731301:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731312:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731338:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.731347:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f817 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.731547:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31009us (31321us total) trans 4294973376 rc 0/0 00000100:00100000:1.0:1789760772.731558:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6088 00000100:00100000:1.0:1789760772.731567:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751134213575 time 751165809915 state 0x2 ref 0 00000100:00100000:1.0:1789760772.735204:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.735304:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776512 00000100:00100000:2.0:1789760772.735314:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776512 00000100:00100000:2.0:1789760772.735366:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751134213575 time 751165809915 state 0x0 ref 1 00000100:00100000:2.0:1789760772.735381:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751166070331 time 751165809915 state 0x1 ref 1 00000100:00100000:2.0:1789760772.735499:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751166070331 time 751200787384 state 0x0 ref 1 00000100:00100000:2.0:1789760772.735513:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.735558:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24821760, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.735566:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.735581:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.735586:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.735593:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.735606:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.766891:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24821760 -> 24825856 00010000:00010000:2.0:1789760772.766911:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766924:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766934:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766943:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760772.766949:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766958:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766981:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766990:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f81e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.767095:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515776512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31578us (31893us total) trans 4294973377 rc 0/0 00000100:00100000:2.0:1789760772.767106:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6089 00000100:00100000:2.0:1789760772.767116:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751166070331 time 751200787384 state 0x2 ref 0 00000001:00080000:1.0:1789760772.773478:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973377 is committed 00000100:00100000:0.0:1789760772.775919:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.776016:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776640 00000100:00100000:2.0:1789760772.776026:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776640 00000100:00100000:2.0:1789760772.776083:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751166070331 time 751200787384 state 0x0 ref 1 00000100:00100000:2.0:1789760772.776100:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751201047800 time 751200787384 state 0x1 ref 1 00000100:00100000:2.0:1789760772.776168:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751201047800 time 751241456940 state 0x0 ref 1 00000100:00100000:2.0:1789760772.776182:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.776233:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24825856, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:2.0:1789760772.776242:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.776258:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.776263:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.776270:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.776285:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.823952:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24825856 -> 24829952 00010000:00010000:1.0:1789760772.823971:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.823984:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.823994:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.824003:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.824009:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.824019:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.824039:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.824047:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1f825 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.824140:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515776640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47956us (48228us total) trans 4294973378 rc 0/0 00000100:00100000:1.0:1789760772.824151:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6090 00000100:00100000:1.0:1789760772.824161:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751201047800 time 751241456940 state 0x2 ref 0 00000001:00080000:3.0:1789760772.825608:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973378 is committed 00000100:00100000:0.0:1789760772.827206:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760772.827281:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776768 00000100:00100000:3.0:1789760772.827291:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776768 00000100:00100000:3.0:1789760772.827341:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751201047800 time 751241456940 state 0x0 ref 1 00000100:00100000:3.0:1789760772.827355:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751241717356 time 751241456940 state 0x1 ref 1 00000100:00100000:3.0:1789760772.827385:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751241717356 time 751292674536 state 0x0 ref 1 00000100:00100000:3.0:1789760772.827408:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760772.827455:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24829952, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xc007 00010000:00010000:3.0:1789760772.827465:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.827480:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.827484:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.827491:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.827505:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.854653:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973379 is committed 00002000:00010000:3.0:1789760772.854672:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24829952 -> 24834048 00002000:00010000:3.0:1789760772.854676:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:3.0:1789760772.854679:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00010000:00010000:3.0:1789760772.854692:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.854735:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.854759:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.854767:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760772.854773:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.854781:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.854801:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.854810:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f82c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.854899:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515776768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27489us (27698us total) trans 4294973379 rc 0/0 00000100:00100000:3.0:1789760772.854908:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6091 00000100:00100000:3.0:1789760772.854917:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751241717356 time 751292674536 state 0x2 ref 0 00000100:00100000:0.0:1789760772.858431:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.858765:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515776896 00000100:00100000:1.0:1789760772.858776:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515776896 00000100:00100000:1.0:1789760772.858834:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751241717356 time 751292674536 state 0x0 ref 1 00000100:00100000:1.0:1789760772.858850:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751292934952 time 751292674536 state 0x1 ref 1 00000100:00100000:1.0:1789760772.858882:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751292934952 time 751324170474 state 0x0 ref 1 00000100:00100000:1.0:1789760772.858894:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515776896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.858941:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24834048, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760772.858950:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.858968:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.858973:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.858979:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.858992:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.881106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973380 is committed 00002000:00010000:1.0:1789760772.881193:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24834048 -> 24838144 00010000:00010000:1.0:1789760772.881210:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.881223:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.881233:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.881241:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.881245:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.881255:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.881273:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.881282:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f833 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.881419:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515776896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22522us (22992us total) trans 4294973380 rc 0/0 00000100:00100000:1.0:1789760772.881430:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6092 00000100:00100000:1.0:1789760772.881440:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751292934952 time 751324170474 state 0x2 ref 0 00000100:00100000:0.0:1789760772.885049:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.885188:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777024 00000100:00100000:2.0:1789760772.885198:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777024 00000100:00100000:2.0:1789760772.885249:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751292934952 time 751324170474 state 0x0 ref 1 00000100:00100000:2.0:1789760772.885265:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751324430890 time 751324170474 state 0x1 ref 1 00000100:00100000:2.0:1789760772.885327:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751324430890 time 751350615703 state 0x0 ref 1 00000100:00100000:2.0:1789760772.885340:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515777024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.885388:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24838144, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:2.0:1789760772.885408:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.885425:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.885429:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.885436:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.885452:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.908047:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973381 is committed 00002000:00010000:1.0:1789760772.908155:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24838144 -> 24842240 00010000:00010000:1.0:1789760772.908169:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.908182:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.908190:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.908197:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.908202:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.908211:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.908236:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.908244:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1f83a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.908344:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515777024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23000us (23298us total) trans 4294973381 rc 0/0 00000100:00100000:1.0:1789760772.908353:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6093 00000100:00100000:1.0:1789760772.908364:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751324430890 time 751350615703 state 0x2 ref 0 00000100:00100000:0.0:1789760772.913825:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.913889:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777152 00000100:00100000:1.0:1789760772.913900:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777152 00000100:00100000:1.0:1789760772.913961:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751324430890 time 751350615703 state 0x0 ref 1 00000100:00100000:1.0:1789760772.913976:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751350876119 time 751350615703 state 0x1 ref 1 00000100:00100000:1.0:1789760772.914004:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751350876119 time 751379293446 state 0x0 ref 1 00000100:00100000:1.0:1789760772.914016:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515777152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.914069:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24842240, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760772.914079:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.914094:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.914099:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.914110:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.914124:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.954557:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24842240 -> 24846336 00010000:00010000:1.0:1789760772.954583:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.954598:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.954608:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.954615:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760772.954621:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.954631:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.954650:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.954662:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f841 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.954686:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973382 is committed 00000100:00100000:1.0:1789760772.954956:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515777152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40936us (41135us total) trans 4294973382 rc 0/0 00000100:00100000:1.0:1789760772.954967:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6094 00000100:00100000:1.0:1789760772.954977:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751350876119 time 751379293446 state 0x2 ref 0 00000100:00100000:0.0:1789760772.958112:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760772.958304:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777280 00000100:00100000:2.0:1789760772.958314:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777280 00000100:00100000:2.0:1789760772.958369:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751350876119 time 751379293446 state 0x0 ref 1 00000100:00100000:2.0:1789760772.958383:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751379553862 time 751379293446 state 0x1 ref 1 00000100:00100000:2.0:1789760772.958447:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751379553862 time 751423735463 state 0x0 ref 1 00000100:00100000:2.0:1789760772.958459:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515777280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760772.958507:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24846336, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:2.0:1789760772.958515:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.958530:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.958534:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.958540:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.958554:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.992503:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973383 is committed 00002000:00010000:0.0:1789760772.992611:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24846336 -> 24850432 00010000:00010000:0.0:1789760772.992632:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.992646:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.992654:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.992660:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760772.992664:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.992684:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.992737:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.992744:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1f848 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.992916:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515777280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34452us (34807us total) trans 4294973383 rc 0/0 00000100:00100000:0.0:1789760772.992929:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6095 00000100:00100000:0.0:1789760772.992939:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751379553862 time 751423735463 state 0x2 ref 0 00000100:00100000:0.0:1789760772.997801:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760772.997973:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777408 00000100:00100000:1.0:1789760772.997984:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777408 00000100:00100000:1.0:1789760772.998040:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751379553862 time 751423735463 state 0x0 ref 1 00000100:00100000:1.0:1789760772.998055:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751423995879 time 751423735463 state 0x1 ref 1 00000100:00100000:1.0:1789760772.998087:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751423995879 time 751463376389 state 0x0 ref 1 00000100:00100000:1.0:1789760772.998260:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515777408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760772.998304:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24850432, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760772.998312:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.998327:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.998331:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.998337:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.998356:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.043981:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.044025:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.044060:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760773.044128:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777536 00000100:00100000:3.0:1789760773.044138:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777536 00000100:00100000:3.0:1789760773.044193:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 404 00000100:00100000:3.0:1789760773.044203:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515777536:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760773.044269:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515777536:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 67us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.044276:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 404 00000100:00100000:3.0:1789760773.044309:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777792 00000100:00100000:3.0:1789760773.044313:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777792 00000100:00100000:3.0:1789760773.044348:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777664 00000100:00100000:3.0:1789760773.044351:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777664 00000100:00100000:3.0:1789760773.044375:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 486 00000100:00100000:3.0:1789760773.044382:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515777792:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760773.044413:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057bbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515777792:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 31us (388us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.044420:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 486 00000100:00100000:3.0:1789760773.044432:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 487 00000100:00100000:3.0:1789760773.044439:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515777664:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760773.044462:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515777664:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 24us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.044469:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 487 00000100:00080000:3.0:1789760773.044484:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:2.0:1789760773.054830:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24850432 -> 24854528 00010000:00010000:2.0:1789760773.054850:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.054862:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.054871:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.054878:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760773.054883:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.054892:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.054911:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.054920:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1f84f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.055048:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515777408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56788us (57252us total) trans 4294973384 rc 0/0 00000100:00100000:2.0:1789760773.055059:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6096 00000100:00100000:2.0:1789760773.055068:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751423995879 time 751463376389 state 0x2 ref 0 00000001:00080000:3.0:1789760773.057062:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973384 is committed 00000100:00100000:0.0:1789760773.060172:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760773.060253:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515777920 00000100:00100000:3.0:1789760773.060266:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515777920 00000100:00100000:3.0:1789760773.060321:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751423995879 time 751463376389 state 0x0 ref 1 00000100:00100000:3.0:1789760773.060337:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751463636805 time 751463376389 state 0x1 ref 1 00000100:00100000:3.0:1789760773.060369:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751463636805 time 751525658730 state 0x0 ref 1 00000100:00100000:3.0:1789760773.060381:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515777920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760773.060453:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24854528, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:3.0:1789760773.060463:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.060480:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760773.060485:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.060491:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.060506:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760773.102396:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24854528 -> 24858624 00010000:00010000:3.0:1789760773.102420:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102435:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102444:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102453:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760773.102458:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102467:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102485:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.102493:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f856 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.102579:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515777920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42195us (42411us total) trans 4294973385 rc 0/0 00000100:00100000:3.0:1789760773.102588:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6097 00000100:00100000:3.0:1789760773.102599:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751463636805 time 751525658730 state 0x2 ref 0 00000001:00080000:3.0:1789760773.102632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973385 is committed 00000100:00100000:0.0:1789760773.108216:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.108376:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778048 00000100:00100000:1.0:1789760773.108386:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778048 00000100:00100000:1.0:1789760773.108440:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751463636805 time 751525658730 state 0x0 ref 1 00000100:00100000:1.0:1789760773.108456:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751525919146 time 751525658730 state 0x1 ref 1 00000100:00100000:1.0:1789760773.108487:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751525919146 time 751573776577 state 0x0 ref 1 00000100:00100000:1.0:1789760773.108501:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515778048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.108559:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24858624, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.108568:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.108583:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760773.108588:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.108595:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.108608:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.151361:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973386 is committed 00002000:00010000:1.0:1789760773.151544:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24858624 -> 24862720 00010000:00010000:1.0:1789760773.151560:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.151580:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.151589:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.151597:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.151602:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.151611:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.151630:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.151638:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f85d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.152299:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515778048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43794us (44085us total) trans 4294973386 rc 0/0 00000100:00100000:1.0:1789760773.152311:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6098 00000100:00100000:1.0:1789760773.152321:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751525919146 time 751573776577 state 0x2 ref 0 00000100:00100000:2.0:1789760773.154861:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.155203:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778176 00000100:00100000:1.0:1789760773.155213:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778176 00000100:00100000:1.0:1789760773.155264:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751525919146 time 751573776577 state 0x0 ref 1 00000100:00100000:1.0:1789760773.155280:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751574036993 time 751573776577 state 0x1 ref 1 00000100:00100000:1.0:1789760773.155341:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751574036993 time 751620630262 state 0x0 ref 1 00000100:00100000:1.0:1789760773.155354:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.155407:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24862720, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.155415:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.155430:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760773.155435:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.155441:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.155454:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.198612:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973387 is committed 00002000:00010000:1.0:1789760773.198641:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24862720 -> 24866816 00010000:00010000:1.0:1789760773.198736:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.198861:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.198872:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.198877:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.198880:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.198887:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.198901:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.198907:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f864 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.198990:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43634us (44134us total) trans 4294973387 rc 0/0 00000100:00100000:1.0:1789760773.198997:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6099 00000100:00100000:1.0:1789760773.199004:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751574036993 time 751620630262 state 0x2 ref 0 00000100:00100000:0.0:1789760773.204241:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.204488:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778304 00000100:00100000:1.0:1789760773.204498:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778304 00000100:00100000:1.0:1789760773.204549:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751574036993 time 751620630262 state 0x0 ref 1 00000100:00100000:1.0:1789760773.204564:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751620890678 time 751620630262 state 0x1 ref 1 00000100:00100000:1.0:1789760773.204589:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751620890678 time 751669878768 state 0x0 ref 1 00000100:00100000:1.0:1789760773.204601:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515778304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.204646:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24866816, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.204654:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.204668:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760773.204673:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.204678:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.204691:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.241278:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973388 is committed 00002000:00010000:1.0:1789760773.243100:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24866816 -> 24870912 00010000:00010000:1.0:1789760773.243119:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.243131:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.243140:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.243147:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.243152:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.243161:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.243179:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.243187:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f86b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.243287:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515778304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38684us (39051us total) trans 4294973388 rc 0/0 00000100:00100000:1.0:1789760773.243295:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6100 00000100:00100000:1.0:1789760773.243303:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751620890678 time 751669878768 state 0x2 ref 0 00000100:00100000:0.0:1789760773.249602:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760773.249692:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778432 00000100:00100000:2.0:1789760773.249702:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778432 00000100:00100000:2.0:1789760773.249814:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751620890678 time 751669878768 state 0x0 ref 1 00000100:00100000:2.0:1789760773.249829:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751670139184 time 751669878768 state 0x1 ref 1 00000100:00100000:2.0:1789760773.249862:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751670139184 time 751715150779 state 0x0 ref 1 00000100:00100000:2.0:1789760773.250030:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760773.250077:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24870912, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:2.0:1789760773.250086:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.250102:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760773.250106:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.250112:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.250126:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.301512:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24870912 -> 24875008 00010000:00010000:0.0:1789760773.301531:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.301544:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.301553:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.301560:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760773.301564:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.301573:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.301593:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.301600:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1f872 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.301755:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973389 is committed 00000100:00100000:0.0:1789760773.302022:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51992us (52423us total) trans 4294973389 rc 0/0 00000100:00100000:0.0:1789760773.302034:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6101 00000100:00100000:0.0:1789760773.302044:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751670139184 time 751715150779 state 0x2 ref 0 00000100:00100000:3.0:1789760773.308355:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760773.308984:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778560 00000100:00100000:2.0:1789760773.308999:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778560 00000100:00100000:2.0:1789760773.309055:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751670139184 time 751715150779 state 0x0 ref 1 00000100:00100000:2.0:1789760773.309070:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751715411195 time 751715150779 state 0x1 ref 1 00000100:00100000:2.0:1789760773.309104:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751715411195 time 751774392957 state 0x0 ref 1 00000100:00100000:2.0:1789760773.309116:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760773.309163:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24875008, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:2.0:1789760773.309173:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.309188:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760773.309193:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.309200:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.309220:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.376460:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973390 is committed 00002000:00010000:1.0:1789760773.376677:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24875008 -> 24879104 00010000:00010000:1.0:1789760773.376693:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.376997:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377328:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377345:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.377351:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377360:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377380:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.377389:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1f879 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.377480:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 68362us (69130us total) trans 4294973390 rc 0/0 00000100:00100000:1.0:1789760773.377490:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6102 00000100:00100000:1.0:1789760773.377500:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751715411195 time 751774392957 state 0x2 ref 0 00000100:00100000:3.0:1789760773.381049:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.381145:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778688 00000100:00100000:0.0:1789760773.381156:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778688 00000100:00100000:0.0:1789760773.381209:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751715411195 time 751774392957 state 0x0 ref 1 00000100:00100000:0.0:1789760773.381225:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751774653373 time 751774392957 state 0x1 ref 1 00000100:00100000:0.0:1789760773.381258:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751774653373 time 751846546768 state 0x0 ref 1 00000100:00100000:0.0:1789760773.381271:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515778688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760773.381317:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24879104, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:0.0:1789760773.381326:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.381342:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760773.381346:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.381352:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.381366:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.440425:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760773.443519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973391 is committed 00002000:00010000:1.0:1789760773.443918:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24879104 -> 24883200 00010000:00010000:1.0:1789760773.443937:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.443949:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.443962:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.443968:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.443973:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.443980:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.443998:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.444005:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f880 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.444197:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515778688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62924us (63152us total) trans 4294973391 rc 0/0 00000100:00100000:1.0:1789760773.444208:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6103 00000100:00100000:1.0:1789760773.444217:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751774653373 time 751846546768 state 0x2 ref 0 00000100:00100000:2.0:1789760773.452443:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.452531:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778816 00000100:00100000:1.0:1789760773.452541:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778816 00000100:00100000:1.0:1789760773.452594:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751774653373 time 751846546768 state 0x0 ref 1 00000100:00100000:1.0:1789760773.452609:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751846807184 time 751846546768 state 0x1 ref 1 00000100:00100000:1.0:1789760773.452644:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751846807184 time 751917933019 state 0x0 ref 1 00000100:00100000:1.0:1789760773.452656:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515778816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.452983:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24883200, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.452992:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.453008:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760773.453012:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.453018:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.453032:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.485224:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973392 is committed 00002000:00010000:1.0:1789760773.485324:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24883200 -> 24887296 00010000:00010000:1.0:1789760773.485339:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.485351:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.485360:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.485367:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.485372:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.485380:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.485398:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.485406:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f887 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.485498:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515778816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32839us (33059us total) trans 4294973392 rc 0/0 00000100:00100000:1.0:1789760773.485508:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6104 00000100:00100000:1.0:1789760773.485518:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751846807184 time 751917933019 state 0x2 ref 0 00000100:00100000:3.0:1789760773.491434:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.492209:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515778944 00000100:00100000:1.0:1789760773.492219:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515778944 00000100:00100000:1.0:1789760773.492273:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751846807184 time 751917933019 state 0x0 ref 1 00000100:00100000:1.0:1789760773.492287:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751918193435 time 751917933019 state 0x1 ref 1 00000100:00100000:1.0:1789760773.492444:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751918193435 time 751957730823 state 0x0 ref 1 00000100:00100000:1.0:1789760773.492457:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.492504:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24887296, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.492514:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.492532:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760773.492536:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.492542:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.492556:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.529122:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973393 is committed 00002000:00010000:0.0:1789760773.529422:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24887296 -> 24891392 00010000:00010000:0.0:1789760773.529438:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.529450:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.529476:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.529485:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760773.529490:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.529504:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.529522:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.529529:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f88e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.529615:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515778944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37156us (38187us total) trans 4294973393 rc 0/0 00000100:00100000:0.0:1789760773.529625:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6105 00000100:00100000:0.0:1789760773.529639:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751918193435 time 751957730823 state 0x2 ref 0 00000100:00100000:3.0:1789760773.533081:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.533190:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779072 00000100:00100000:1.0:1789760773.533201:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779072 00000100:00100000:1.0:1789760773.533251:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751918193435 time 751957730823 state 0x0 ref 1 00000100:00100000:1.0:1789760773.533266:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751957991239 time 751957730823 state 0x1 ref 1 00000100:00100000:1.0:1789760773.533330:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751957991239 time 751998618934 state 0x0 ref 1 00000100:00100000:1.0:1789760773.533343:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.533389:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24891392, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.533397:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.533413:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760773.533419:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.533425:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.533440:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.560485:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973394 is committed 00002000:00010000:1.0:1789760773.560936:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24891392 -> 24895488 00010000:00010000:1.0:1789760773.560951:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560965:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560974:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560982:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.560987:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560997:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.561020:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.561028:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f895 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.561211:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27865us (28133us total) trans 4294973394 rc 0/0 00000100:00100000:1.0:1789760773.561223:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6106 00000100:00100000:1.0:1789760773.561233:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751957991239 time 751998618934 state 0x2 ref 0 00000100:00100000:2.0:1789760773.565890:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.565970:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779200 00000100:00100000:0.0:1789760773.565980:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779200 00000100:00100000:0.0:1789760773.566033:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751957991239 time 751998618934 state 0x0 ref 1 00000100:00100000:0.0:1789760773.566063:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751998879350 time 751998618934 state 0x1 ref 1 00000100:00100000:0.0:1789760773.566121:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751998879350 time 752031408959 state 0x0 ref 1 00000100:00100000:0.0:1789760773.566135:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760773.566191:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24895488, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:0.0:1789760773.566200:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.566217:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.566222:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.566228:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.566242:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.616291:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973395 is committed 00002000:00010000:0.0:1789760773.616428:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24895488 -> 24899584 00010000:00010000:0.0:1789760773.616444:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.616457:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.616466:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.616479:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760773.616484:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.616493:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.616512:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.616520:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f89c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.616653:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50515us (50950us total) trans 4294973395 rc 0/0 00000100:00100000:0.0:1789760773.616898:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6107 00000100:00100000:0.0:1789760773.616909:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751998879350 time 752031408959 state 0x2 ref 0 00000100:00100000:3.0:1789760773.622056:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760773.622138:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779328 00000100:00100000:2.0:1789760773.622153:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779328 00000100:00100000:2.0:1789760773.622214:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 751998879350 time 752031408959 state 0x0 ref 1 00000100:00100000:2.0:1789760773.622231:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752031669375 time 752031408959 state 0x1 ref 1 00000100:00100000:2.0:1789760773.622289:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752031669375 time 752087578730 state 0x0 ref 1 00000100:00100000:2.0:1789760773.622310:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760773.622355:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24899584, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:2.0:1789760773.622369:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.622384:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.622388:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.622394:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.622407:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.664210:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973396 is committed 00002000:00010000:1.0:1789760773.664317:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24899584 -> 24903680 00010000:00010000:1.0:1789760773.664337:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.664361:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.664370:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.664377:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.664381:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.664504:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.664526:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.664540:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1f8a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.664699:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42386us (42653us total) trans 4294973396 rc 0/0 00000100:00100000:1.0:1789760773.664763:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6108 00000100:00100000:1.0:1789760773.664773:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752031669375 time 752087578730 state 0x2 ref 0 00000100:00100000:3.0:1789760773.668638:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.669793:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779456 00000100:00100000:0.0:1789760773.669805:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779456 00000100:00100000:1.0:1789760773.669978:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752031669375 time 752087578730 state 0x0 ref 1 00000100:00100000:1.0:1789760773.669995:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752087839146 time 752087578730 state 0x1 ref 1 00000100:00100000:1.0:1789760773.670106:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752087839146 time 752135394471 state 0x0 ref 1 00000100:00100000:1.0:1789760773.670120:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515779456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.670168:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24903680, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.670177:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.670192:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.670196:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.670203:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.670217:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.730211:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24903680 -> 24907776 00010000:00010000:1.0:1789760773.730232:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.730245:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.730255:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.730263:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.730268:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.730277:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.730295:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.730303:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.730465:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515779456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60343us (61829us total) trans 4294973397 rc 0/0 00000100:00100000:1.0:1789760773.730479:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6109 00000100:00100000:1.0:1789760773.730488:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752087839146 time 752135394471 state 0x2 ref 0 00000001:00080000:0.0:1789760773.730594:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973397 is committed 00000100:00100000:2.0:1789760773.735985:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.736258:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779584 00000100:00100000:1.0:1789760773.736268:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779584 00000100:00100000:1.0:1789760773.736324:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752087839146 time 752135394471 state 0x0 ref 1 00000100:00100000:1.0:1789760773.736413:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752135654887 time 752135394471 state 0x1 ref 1 00000100:00100000:1.0:1789760773.736444:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752135654887 time 752201733904 state 0x0 ref 1 00000100:00100000:1.0:1789760773.736457:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515779584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.736504:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24907776, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.736513:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.736529:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.736534:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.736540:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.736553:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.770038:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24907776 -> 24911872 00010000:00010000:1.0:1789760773.770058:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.770070:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.770078:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.770085:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760773.770089:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.770098:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.770116:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.770124:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.770210:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973398 is committed 00000100:00100000:1.0:1789760773.770265:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515779584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33804us (34285us total) trans 4294973398 rc 0/0 00000100:00100000:1.0:1789760773.770275:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6110 00000100:00100000:1.0:1789760773.770285:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752135654887 time 752201733904 state 0x2 ref 0 00000100:00100000:2.0:1789760773.776228:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.776375:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779712 00000100:00100000:0.0:1789760773.776385:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779712 00000100:00100000:0.0:1789760773.776432:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752135654887 time 752201733904 state 0x0 ref 1 00000100:00100000:0.0:1789760773.776447:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752201994320 time 752201733904 state 0x1 ref 1 00000100:00100000:0.0:1789760773.776475:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752201994320 time 752241764714 state 0x0 ref 1 00000100:00100000:0.0:1789760773.776486:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515779712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760773.776528:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24911872, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:0.0:1789760773.776536:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.776551:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.776555:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.776561:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.776574:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.813879:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973399 is committed 00002000:00010000:0.0:1789760773.814286:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24911872 -> 24915968 00010000:00010000:0.0:1789760773.814313:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.814325:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.814332:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.814464:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760773.814469:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.814478:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.814497:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.814505:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.814600:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515779712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38111us (38377us total) trans 4294973399 rc 0/0 00000100:00100000:0.0:1789760773.814611:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6111 00000100:00100000:0.0:1789760773.814620:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752201994320 time 752241764714 state 0x2 ref 0 00000100:00100000:2.0:1789760773.836633:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.838671:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779840 00000100:00100000:1.0:1789760773.838685:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779840 00000100:00100000:1.0:1789760773.839040:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752201994320 time 752241764714 state 0x0 ref 1 00000100:00100000:1.0:1789760773.839055:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752242025130 time 752241764714 state 0x1 ref 1 00000100:00100000:1.0:1789760773.839204:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752242025130 time 752304492577 state 0x0 ref 1 00000100:00100000:1.0:1789760773.839219:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515779840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.839271:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24915968, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xc007 00010000:00010000:1.0:1789760773.839280:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.839296:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.839301:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.839306:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.839320:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.889209:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973400 is committed 00002000:00010000:0.0:1789760773.889796:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24915968 -> 24920064 00002000:00010000:0.0:1789760773.889801:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:0.0:1789760773.889804:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00010000:00010000:0.0:1789760773.889817:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889829:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889838:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889845:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760773.889850:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889859:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889879:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.889886:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f8bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.890022:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515779840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50802us (53393us total) trans 4294973400 rc 0/0 00000100:00100000:0.0:1789760773.890032:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6112 00000100:00100000:0.0:1789760773.890042:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752242025130 time 752304492577 state 0x2 ref 0 00000100:00100000:2.0:1789760773.898688:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760773.898840:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515779968 00000100:00100000:0.0:1789760773.898851:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515779968 00000100:00100000:0.0:1789760773.898900:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752242025130 time 752304492577 state 0x0 ref 1 00000100:00100000:0.0:1789760773.898915:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752304752993 time 752304492577 state 0x1 ref 1 00000100:00100000:0.0:1789760773.898946:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752304752993 time 752364235196 state 0x0 ref 1 00000100:00100000:0.0:1789760773.898958:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760773.899007:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24920064, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:0.0:1789760773.899017:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.899033:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.899037:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.899043:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.899056:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.972158:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973401 is committed 00002000:00010000:2.0:1789760773.972246:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24920064 -> 24924160 00010000:00010000:2.0:1789760773.972261:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.972275:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.972285:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.972293:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760773.972298:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.972307:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.972326:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.972334:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1f8c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.972917:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515779968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 73954us (74232us total) trans 4294973401 rc 0/0 00000100:00100000:2.0:1789760773.972928:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6113 00000100:00100000:2.0:1789760773.972939:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752304752993 time 752364235196 state 0x2 ref 0 00000100:00100000:2.0:1789760773.985868:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760773.986029:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780096 00000100:00100000:1.0:1789760773.986040:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780096 00000100:00100000:1.0:1789760773.986090:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752304752993 time 752364235196 state 0x0 ref 1 00000100:00100000:1.0:1789760773.986106:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752364495612 time 752364235196 state 0x1 ref 1 00000100:00100000:1.0:1789760773.986489:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752364495612 time 752451776288 state 0x0 ref 1 00000100:00100000:1.0:1789760773.986508:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515780096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760773.986555:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24924160, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760773.986565:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.986581:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.986586:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.986592:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.986606:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.009209:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973402 is committed 00002000:00010000:2.0:1789760774.009367:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24924160 -> 24928256 00010000:00010000:2.0:1789760774.009382:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.009400:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.009409:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.009416:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760774.009421:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.009429:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.009449:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.009457:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f8cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.011336:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515780096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23043us (23690us total) trans 4294973402 rc 0/0 00000100:00100000:2.0:1789760774.011349:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6114 00000100:00100000:2.0:1789760774.011359:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752364495612 time 752451776288 state 0x2 ref 0 00000100:00100000:0.0:1789760774.015266:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760774.015393:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780224 00000100:00100000:2.0:1789760774.015402:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780224 00000100:00080000:3.0:1789760774.015451:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760774.015469:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752364495612 time 752451776288 state 0x0 ref 1 00000100:00100000:2.0:1789760774.015485:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752452036704 time 752451776288 state 0x1 ref 1 00000100:00100000:2.0:1789760774.015516:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752452036704 time 752480805296 state 0x0 ref 1 00000100:00100000:2.0:1789760774.015529:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515780224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760774.015576:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24928256, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:2.0:1789760774.015583:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.015594:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760774.015598:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.015602:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.015612:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.048447:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24928256 -> 24932352 00010000:00010000:2.0:1789760774.048466:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.048479:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.048492:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.048499:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760774.048503:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.048511:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.048530:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.048537:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.048628:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515780224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33097us (33363us total) trans 4294973403 rc 0/0 00000100:00100000:2.0:1789760774.048647:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6115 00000100:00100000:2.0:1789760774.048805:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752452036704 time 752480805296 state 0x2 ref 0 00000001:00080000:0.0:1789760774.050494:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973403 is committed 00000100:00100000:0.0:1789760774.054914:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.055005:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780352 00000100:00100000:1.0:1789760774.055015:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780352 00000100:00100000:1.0:1789760774.055067:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752452036704 time 752480805296 state 0x0 ref 1 00000100:00100000:1.0:1789760774.055083:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752481065712 time 752480805296 state 0x1 ref 1 00000100:00100000:1.0:1789760774.055154:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752481065712 time 752520442327 state 0x0 ref 1 00000100:00100000:1.0:1789760774.055167:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515780352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.055334:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24932352, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.055352:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.055369:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760774.055374:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.055381:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.055395:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.112629:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24932352 -> 24936448 00010000:00010000:2.0:1789760774.112666:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.112687:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.112699:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.112728:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760774.112733:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.112752:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.112772:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.112780:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f8db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.112831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973404 is committed 00000100:00100000:2.0:1789760774.112862:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515780352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57693us (58070us total) trans 4294973404 rc 0/0 00000100:00100000:2.0:1789760774.112871:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6116 00000100:00100000:2.0:1789760774.112886:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752481065712 time 752520442327 state 0x2 ref 0 00000100:00100000:1.0:1789760774.117931:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760774.118562:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780480 00000100:00100000:0.0:1789760774.118571:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780480 00000100:00100000:0.0:1789760774.118616:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752481065712 time 752520442327 state 0x0 ref 1 00000100:00100000:0.0:1789760774.118626:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752520702743 time 752520442327 state 0x1 ref 1 00000100:00100000:0.0:1789760774.118657:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752520702743 time 752583945898 state 0x0 ref 1 00000100:00100000:0.0:1789760774.118672:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515780480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760774.118755:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24936448, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:0.0:1789760774.118764:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.118781:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760774.118786:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.118792:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.118805:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.144397:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24936448 -> 24940544 00010000:00010000:3.0:1789760774.144417:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.144430:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.144438:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.144446:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760774.144450:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.144459:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.144478:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.144485:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1f8e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.144581:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515780480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25907us (26654us total) trans 4294973405 rc 0/0 00000100:00100000:3.0:1789760774.144592:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6117 00000100:00100000:3.0:1789760774.144601:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752520702743 time 752583945898 state 0x2 ref 0 00000100:00100000:0.0:1789760774.148615:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.148819:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780608 00000100:00100000:1.0:1789760774.148830:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780608 00000100:00100000:1.0:1789760774.148879:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752520702743 time 752583945898 state 0x0 ref 1 00000100:00100000:1.0:1789760774.148892:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752584206314 time 752583945898 state 0x1 ref 1 00000100:00100000:1.0:1789760774.148923:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752584206314 time 752614212056 state 0x0 ref 1 00000100:00100000:1.0:1789760774.148935:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515780608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.148980:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24940544, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.148989:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.149004:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760774.149008:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.149015:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.149030:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.149946:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973405 is committed 00000001:00080000:1.0:1789760774.194936:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973406 is committed 00002000:00010000:0.0:1789760774.194985:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24940544 -> 24944640 00010000:00010000:0.0:1789760774.195002:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.195016:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.195026:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.195033:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760774.195039:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.195048:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.195067:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.195075:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1f8e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.195298:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515780608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46360us (46686us total) trans 4294973406 rc 0/0 00000100:00100000:0.0:1789760774.195309:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6118 00000100:00100000:0.0:1789760774.195318:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752584206314 time 752614212056 state 0x2 ref 0 00000100:00100000:1.0:1789760774.200396:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760774.200470:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780736 00000100:00100000:2.0:1789760774.200479:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780736 00000100:00100000:2.0:1789760774.200527:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752584206314 time 752614212056 state 0x0 ref 1 00000100:00100000:2.0:1789760774.200541:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752614472472 time 752614212056 state 0x1 ref 1 00000100:00100000:2.0:1789760774.200570:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752614472472 time 752665859513 state 0x0 ref 1 00000100:00100000:2.0:1789760774.200583:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515780736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760774.200630:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24944640, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:2.0:1789760774.200639:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.200653:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760774.200658:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.200664:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.200678:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.229489:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973407 is committed 00002000:00010000:1.0:1789760774.229594:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24944640 -> 24948736 00010000:00010000:1.0:1789760774.229610:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.229624:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.229633:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.229641:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.229700:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.229729:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.229758:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.229767:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1f8f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.229853:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515780736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29268us (29468us total) trans 4294973407 rc 0/0 00000100:00100000:1.0:1789760774.229863:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6119 00000100:00100000:1.0:1789760774.229872:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752614472472 time 752665859513 state 0x2 ref 0 00000100:00100000:1.0:1789760774.234201:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760774.234642:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780864 00000100:00100000:2.0:1789760774.234653:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780864 00000100:00100000:2.0:1789760774.234748:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752614472472 time 752665859513 state 0x0 ref 1 00000100:00100000:2.0:1789760774.234765:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752666119929 time 752665859513 state 0x1 ref 1 00000100:00100000:2.0:1789760774.234983:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752666119929 time 752700270331 state 0x0 ref 1 00000100:00100000:2.0:1789760774.234998:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515780864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760774.235055:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24948736, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:2.0:1789760774.235064:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.235088:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760774.235094:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.235100:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.235122:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.281668:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973408 is committed 00002000:00010000:1.0:1789760774.281907:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24948736 -> 24952832 00010000:00010000:1.0:1789760774.281923:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.281936:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.281945:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.281952:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.281958:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.281966:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.281985:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.281998:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1f8f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.282093:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515780864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47092us (47896us total) trans 4294973408 rc 0/0 00000100:00100000:1.0:1789760774.282103:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6120 00000100:00100000:1.0:1789760774.282498:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752666119929 time 752700270331 state 0x2 ref 0 00000100:00100000:0.0:1789760774.290024:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760774.290329:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515780992 00000100:00100000:3.0:1789760774.290340:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515780992 00000100:00100000:3.0:1789760774.290405:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752666119929 time 752700270331 state 0x0 ref 1 00000100:00100000:3.0:1789760774.290420:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752700530747 time 752700270331 state 0x1 ref 1 00000100:00100000:3.0:1789760774.290449:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752700530747 time 752755738495 state 0x0 ref 1 00000100:00100000:3.0:1789760774.290462:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515780992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760774.290511:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24952832, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:3.0:1789760774.290520:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.290535:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760774.290539:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.290545:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.290559:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.322640:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973409 is committed 00002000:00010000:1.0:1789760774.323041:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24952832 -> 24956928 00010000:00010000:1.0:1789760774.323057:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323069:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323077:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323084:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.323088:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323098:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323122:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323130:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1f8fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.323267:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515780992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32801us (33246us total) trans 4294973409 rc 0/0 00000100:00100000:1.0:1789760774.323277:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6121 00000100:00100000:1.0:1789760774.323287:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752700530747 time 752755738495 state 0x2 ref 0 00000100:00100000:2.0:1789760774.332540:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.332689:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781120 00000100:00100000:1.0:1789760774.332699:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781120 00000100:00100000:1.0:1789760774.332797:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752700530747 time 752755738495 state 0x0 ref 1 00000100:00100000:1.0:1789760774.332813:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752755998911 time 752755738495 state 0x1 ref 1 00000100:00100000:1.0:1789760774.332843:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752755998911 time 752798131988 state 0x0 ref 1 00000100:00100000:1.0:1789760774.332855:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515781120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.332906:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24956928, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.332915:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.332932:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760774.332937:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.332943:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.332962:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.373052:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973410 is committed 00002000:00010000:1.0:1789760774.373217:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24956928 -> 24961024 00010000:00010000:1.0:1789760774.373233:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.373246:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.373255:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.373263:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.373269:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.373277:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.373295:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.373303:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f905 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.373399:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515781120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40542us (40865us total) trans 4294973410 rc 0/0 00000100:00100000:1.0:1789760774.373876:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6122 00000100:00100000:1.0:1789760774.373884:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752755998911 time 752798131988 state 0x2 ref 0 00000100:00100000:2.0:1789760774.383079:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760774.383169:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781248 00000100:00100000:3.0:1789760774.383178:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781248 00000100:00100000:3.0:1789760774.383228:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752755998911 time 752798131988 state 0x0 ref 1 00000100:00100000:3.0:1789760774.383244:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752798392404 time 752798131988 state 0x1 ref 1 00000100:00100000:3.0:1789760774.383273:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752798392404 time 752848562956 state 0x0 ref 1 00000100:00100000:3.0:1789760774.383284:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515781248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760774.383331:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24961024, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:3.0:1789760774.383340:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383356:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760774.383361:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.383367:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.383393:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.430857:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24961024 -> 24965120 00010000:00010000:3.0:1789760774.430878:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.430892:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.430900:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.430907:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760774.430912:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.430921:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.430940:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.430948:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f90c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.431041:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515781248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47746us (47959us total) trans 4294973411 rc 0/0 00000100:00100000:3.0:1789760774.431051:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6123 00000100:00100000:3.0:1789760774.431061:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752798392404 time 752848562956 state 0x2 ref 0 00000001:00080000:0.0:1789760774.431602:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973411 is committed 00000100:00100000:0.0:1789760774.437563:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.437670:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781376 00000100:00100000:1.0:1789760774.437680:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781376 00000100:00100000:1.0:1789760774.437774:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752798392404 time 752848562956 state 0x0 ref 1 00000100:00100000:1.0:1789760774.437790:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752848823372 time 752848562956 state 0x1 ref 1 00000100:00100000:1.0:1789760774.437825:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752848823372 time 752903113697 state 0x0 ref 1 00000100:00100000:1.0:1789760774.437837:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937caac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515781376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.437885:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24965120, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.437894:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.437911:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760774.437915:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.437921:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.437934:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.462269:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760774.477637:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973412 is committed 00002000:00010000:0.0:1789760774.478951:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24965120 -> 24969216 00010000:00010000:0.0:1789760774.478968:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.478980:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.478994:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.479002:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760774.479007:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.479015:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.479031:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.479038:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1f913 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.479158:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937caac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515781376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41318us (41600us total) trans 4294973412 rc 0/0 00000100:00100000:0.0:1789760774.479168:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6124 00000100:00100000:0.0:1789760774.479176:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752848823372 time 752903113697 state 0x2 ref 0 00000100:00100000:2.0:1789760774.484186:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760774.484235:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781504 00000100:00100000:3.0:1789760774.484245:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781504 00000100:00100000:3.0:1789760774.484297:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752848823372 time 752903113697 state 0x0 ref 1 00000100:00100000:3.0:1789760774.484312:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752903374113 time 752903113697 state 0x1 ref 1 00000100:00100000:3.0:1789760774.484401:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752903374113 time 752949688708 state 0x0 ref 1 00000100:00100000:3.0:1789760774.484416:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515781504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760774.484469:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24969216, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:3.0:1789760774.484478:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.484493:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760774.484497:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.484503:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.484518:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.514571:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24969216 -> 24973312 00010000:00010000:1.0:1789760774.514591:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.514603:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.514612:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.514619:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.514624:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.514645:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.514672:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.514681:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1f91a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.515008:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515781504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30589us (30826us total) trans 4294973413 rc 0/0 00000100:00100000:1.0:1789760774.515020:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6125 00000100:00100000:1.0:1789760774.515029:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752903374113 time 752949688708 state 0x2 ref 0 00000001:00080000:2.0:1789760774.515574:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973413 is committed 00000100:00100000:3.0:1789760774.525183:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.525283:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781632 00000100:00100000:1.0:1789760774.525293:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781632 00000100:00100000:1.0:1789760774.525344:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752903374113 time 752949688708 state 0x0 ref 1 00000100:00100000:1.0:1789760774.525360:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752949949124 time 752949688708 state 0x1 ref 1 00000100:00100000:1.0:1789760774.525435:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752949949124 time 752990722436 state 0x0 ref 1 00000100:00100000:1.0:1789760774.525449:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515781632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.525497:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24973312, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.525506:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.525519:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760774.525524:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.525530:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.525542:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.563353:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973414 is committed 00002000:00010000:1.0:1789760774.563452:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24973312 -> 24977408 00010000:00010000:1.0:1789760774.563469:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.563482:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.563491:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.563499:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.563504:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.563514:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.563535:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.563543:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f921 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.563638:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515781632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38187us (38460us total) trans 4294973414 rc 0/0 00000100:00100000:1.0:1789760774.563648:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6126 00000100:00100000:1.0:1789760774.563667:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752949949124 time 752990722436 state 0x2 ref 0 00000100:00100000:0.0:1789760774.575348:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760774.575413:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781760 00000100:00100000:3.0:1789760774.575424:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781760 00000100:00100000:3.0:1789760774.575550:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752949949124 time 752990722436 state 0x0 ref 1 00000100:00100000:3.0:1789760774.575566:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752990982852 time 752990722436 state 0x1 ref 1 00000100:00100000:3.0:1789760774.575597:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752990982852 time 753040886541 state 0x0 ref 1 00000100:00100000:3.0:1789760774.575610:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515781760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760774.575671:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24977408, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:3.0:1789760774.575681:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.575698:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760774.575703:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.575979:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.575995:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.630074:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24977408 -> 24981504 00010000:00010000:0.0:1789760774.630094:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.630107:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.630116:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.630124:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760774.630129:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.630138:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.630157:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.630165:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1f928 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.630258:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515781760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54645us (54913us total) trans 4294973415 rc 0/0 00000100:00100000:0.0:1789760774.630268:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6127 00000100:00100000:0.0:1789760774.630277:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752990982852 time 753040886541 state 0x2 ref 0 00000001:00080000:0.0:1789760774.630315:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973415 is committed 00000100:00100000:3.0:1789760774.639048:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.639341:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515781888 00000100:00100000:1.0:1789760774.639352:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515781888 00000100:00100000:1.0:1789760774.639532:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 752990982852 time 753040886541 state 0x0 ref 1 00000100:00100000:1.0:1789760774.639550:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753041146957 time 753040886541 state 0x1 ref 1 00000100:00100000:1.0:1789760774.639623:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753041146957 time 753104911272 state 0x0 ref 1 00000100:00100000:1.0:1789760774.639637:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515781888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.639698:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24981504, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.639766:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.639782:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.639787:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.639793:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.639808:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.654202:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8010e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544418816:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760774.654273:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.654562:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418816 00000100:00100000:1.0:1789760774.654570:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418816 00000100:00100000:1.0:1789760774.655300:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789760774.655314:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544418816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760774.655338:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3833593856 avail 3809476608 00000100:00100000:1.0:1789760774.655408:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544418816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 98us (1135us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.655419:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789760774.655489:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8056a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544418944:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760774.655525:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.655565:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544418944 00000100:00100000:1.0:1789760774.655570:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544418944 00000100:00080000:3.0:1789760774.657360:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760774.657389:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760774.657868:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8010e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544418816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760774.658637:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789760774.658732:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544418944:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760774.658781:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760774.658856:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544418944:12345-0@lo:13:osp-pre-1-0.0 Request processed in 211us (3330us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.658864:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789760774.658922:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8056a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544418944:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760774.730793:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973416 is committed 00002000:00010000:1.0:1789760774.731858:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24981504 -> 24985600 00010000:00010000:1.0:1789760774.731876:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.731889:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.731897:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.731904:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760774.731909:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.731917:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.731935:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.731942:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f92f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.732037:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515781888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 92398us (92994us total) trans 4294973416 rc 0/0 00000100:00100000:1.0:1789760774.732048:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6128 00000100:00100000:1.0:1789760774.732058:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753041146957 time 753104911272 state 0x2 ref 0 00000100:00100000:3.0:1789760774.738365:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.738501:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782016 00000100:00100000:1.0:1789760774.738512:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782016 00000100:00100000:1.0:1789760774.738573:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753041146957 time 753104911272 state 0x0 ref 1 00000100:00100000:1.0:1789760774.738590:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753105171688 time 753104911272 state 0x1 ref 1 00000100:00100000:1.0:1789760774.738625:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753105171688 time 753203913304 state 0x0 ref 1 00000100:00100000:1.0:1789760774.738637:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515782016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.738680:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24985600, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:1.0:1789760774.738689:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.738740:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.738744:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.738751:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.738764:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.828177:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973417 is committed 00002000:00010000:3.0:1789760774.828271:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24985600 -> 24989696 00010000:00010000:3.0:1789760774.828288:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.828300:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.828316:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.828324:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760774.828329:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.828339:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.828359:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.828369:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f936 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.829361:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515782016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 90716us (90997us total) trans 4294973417 rc 0/0 00000100:00100000:3.0:1789760774.829374:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6129 00000100:00100000:3.0:1789760774.829384:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753105171688 time 753203913304 state 0x2 ref 0 00000100:00100000:0.0:1789760774.835328:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760774.835954:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782144 00000100:00100000:2.0:1789760774.835964:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782144 00000100:00100000:2.0:1789760774.836019:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753105171688 time 753203913304 state 0x0 ref 1 00000100:00100000:2.0:1789760774.836034:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753204173720 time 753203913304 state 0x1 ref 1 00000100:00100000:2.0:1789760774.836102:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753204173720 time 753301389933 state 0x0 ref 1 00000100:00100000:2.0:1789760774.836116:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515782144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760774.836161:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24989696, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xc007 00010000:00010000:2.0:1789760774.836170:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.836185:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.836202:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.836208:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.836222:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.884880:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973418 is committed 00002000:00010000:0.0:1789760774.884987:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24989696 -> 24993792 00002000:00010000:0.0:1789760774.884992:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:0.0:1789760774.884995:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00010000:00010000:0.0:1789760774.885008:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.885021:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.885029:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.885036:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760774.885041:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.885050:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.885070:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.885078:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1f93d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.885174:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515782144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49056us (49853us total) trans 4294973418 rc 0/0 00000100:00100000:0.0:1789760774.885185:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6130 00000100:00100000:0.0:1789760774.885194:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753204173720 time 753301389933 state 0x2 ref 0 00000100:00100000:2.0:1789760774.888493:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760774.888626:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782272 00000100:00100000:3.0:1789760774.888635:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782272 00000100:00100000:3.0:1789760774.888887:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753204173720 time 753301389933 state 0x0 ref 1 00000100:00100000:3.0:1789760774.888906:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753301650349 time 753301389933 state 0x1 ref 1 00000100:00100000:3.0:1789760774.888938:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753301650349 time 753354227161 state 0x0 ref 1 00000100:00100000:3.0:1789760774.888951:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515782272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760774.889005:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24993792, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:3.0:1789760774.889016:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.889032:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760774.889036:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.889044:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.889057:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.927300:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24993792 -> 24997888 00010000:00010000:3.0:1789760774.927320:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.927333:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.927342:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.927354:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760774.927359:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.927367:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.927460:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.927470:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f944 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.927564:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515782272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38610us (39075us total) trans 4294973419 rc 0/0 00000100:00100000:3.0:1789760774.927574:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6131 00000100:00100000:3.0:1789760774.927583:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753301650349 time 753354227161 state 0x2 ref 0 00000001:00080000:2.0:1789760774.927670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973419 is committed 00000100:00100000:3.0:1789760774.936637:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760774.937298:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782400 00000100:00100000:2.0:1789760774.937309:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782400 00000100:00100000:2.0:1789760774.937367:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753301650349 time 753354227161 state 0x0 ref 1 00000100:00100000:2.0:1789760774.937383:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753354487577 time 753354227161 state 0x1 ref 1 00000100:00100000:2.0:1789760774.937425:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753354487577 time 753402714267 state 0x0 ref 1 00000100:00100000:2.0:1789760774.937442:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515782400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760774.937494:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 24997888, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:2.0:1789760774.937503:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.937519:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.937523:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.937530:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.937543:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.970103:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973420 is committed 00002000:00010000:0.0:1789760774.970398:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 24997888 -> 25001984 00010000:00010000:0.0:1789760774.970414:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.970427:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.970436:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.970444:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760774.970449:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.970458:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.970478:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.970492:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f94b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.970583:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515782400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33143us (33952us total) trans 4294973420 rc 0/0 00000100:00100000:0.0:1789760774.970593:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6132 00000100:00100000:0.0:1789760774.970604:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753354487577 time 753402714267 state 0x2 ref 0 00000100:00100000:0.0:1789760774.974830:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760774.974899:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782528 00000100:00100000:1.0:1789760774.974910:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782528 00000100:00100000:1.0:1789760774.974962:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753354487577 time 753402714267 state 0x0 ref 1 00000100:00100000:1.0:1789760774.974978:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753402974683 time 753402714267 state 0x1 ref 1 00000100:00100000:1.0:1789760774.975010:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753402974683 time 753440298615 state 0x0 ref 1 00000100:00100000:1.0:1789760774.975022:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515782528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760774.975068:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25001984, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760774.975076:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.975091:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.975095:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.975102:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.975121:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.024252:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973421 is committed 00002000:00010000:0.0:1789760775.024794:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25001984 -> 25006080 00010000:00010000:0.0:1789760775.024809:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.024820:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.024829:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.024837:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.024842:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.024850:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.024874:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.024882:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f952 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.024975:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515782528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49950us (50317us total) trans 4294973421 rc 0/0 00000100:00100000:0.0:1789760775.024984:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6133 00000100:00100000:0.0:1789760775.024994:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753402974683 time 753440298615 state 0x2 ref 0 00000100:00100000:2.0:1789760775.029200:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.029337:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782656 00000100:00100000:1.0:1789760775.029348:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782656 00000100:00100000:1.0:1789760775.029403:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753402974683 time 753440298615 state 0x0 ref 1 00000100:00100000:1.0:1789760775.029420:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753440559031 time 753440298615 state 0x1 ref 1 00000100:00100000:1.0:1789760775.029450:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753440559031 time 753494738615 state 0x0 ref 1 00000100:00100000:1.0:1789760775.029463:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515782656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.029520:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25006080, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.029530:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.029547:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760775.029552:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.029559:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.029575:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.087017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973422 is committed 00002000:00010000:0.0:1789760775.087179:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25006080 -> 25010176 00010000:00010000:0.0:1789760775.087194:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.087206:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.087215:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.087223:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.087227:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.087243:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.087263:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.087271:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1f959 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.087365:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515782656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57899us (58171us total) trans 4294973422 rc 0/0 00000100:00100000:0.0:1789760775.087376:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6134 00000100:00100000:0.0:1789760775.087385:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753440559031 time 753494738615 state 0x2 ref 0 00000100:00100000:3.0:1789760775.090297:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.090398:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782784 00000100:00100000:1.0:1789760775.090408:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782784 00000100:00100000:1.0:1789760775.090461:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753440559031 time 753494738615 state 0x0 ref 1 00000100:00100000:1.0:1789760775.090477:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753494999031 time 753494738615 state 0x1 ref 1 00000100:00100000:1.0:1789760775.090508:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753494999031 time 753555796678 state 0x0 ref 1 00000100:00100000:1.0:1789760775.090521:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515782784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.090568:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25010176, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.090576:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.090592:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760775.090597:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.090604:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.090619:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.192165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973423 is committed 00002000:00010000:1.0:1789760775.192334:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25010176 -> 25014272 00010000:00010000:1.0:1789760775.192350:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192364:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192379:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192387:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.192392:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192402:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192423:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192431:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f960 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.192525:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515782784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 102001us (102233us total) trans 4294973423 rc 0/0 00000100:00100000:1.0:1789760775.192536:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6135 00000100:00100000:1.0:1789760775.192547:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753494999031 time 753555796678 state 0x2 ref 0 00000100:00100000:3.0:1789760775.198590:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.198683:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515782912 00000100:00100000:0.0:1789760775.198693:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515782912 00000100:00100000:0.0:1789760775.198780:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753494999031 time 753555796678 state 0x0 ref 1 00000100:00100000:0.0:1789760775.198795:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753556057094 time 753555796678 state 0x1 ref 1 00000100:00100000:0.0:1789760775.198818:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753556057094 time 753664107279 state 0x0 ref 1 00000100:00100000:0.0:1789760775.198830:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515782912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760775.198878:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25014272, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:0.0:1789760775.198886:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.198902:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760775.198906:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.198912:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.198926:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760775.232928:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25014272 -> 25018368 00010000:00010000:2.0:1789760775.232948:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.232960:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.232969:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.232976:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760775.232980:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.232989:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233006:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233012:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f967 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.233324:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973424 is committed 00000100:00100000:2.0:1789760775.233374:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515782912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34542us (34789us total) trans 4294973424 rc 0/0 00000100:00100000:2.0:1789760775.233385:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6136 00000100:00100000:2.0:1789760775.233395:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753556057094 time 753664107279 state 0x2 ref 0 00000100:00100000:2.0:1789760775.238571:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760775.238940:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783040 00000100:00100000:3.0:1789760775.238950:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783040 00000100:00100000:3.0:1789760775.239001:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753556057094 time 753664107279 state 0x0 ref 1 00000100:00100000:3.0:1789760775.239022:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753664367695 time 753664107279 state 0x1 ref 1 00000100:00100000:3.0:1789760775.239042:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753664367695 time 753704331745 state 0x0 ref 1 00000100:00100000:3.0:1789760775.239054:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760775.239109:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25018368, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:3.0:1789760775.239118:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.239133:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760775.239137:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.239142:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.239154:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.282133:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25018368 -> 25022464 00010000:00010000:1.0:1789760775.282155:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.282175:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.282196:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.282205:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.282210:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.282219:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.282239:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.282247:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1f96e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.282340:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515783040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43282us (43774us total) trans 4294973425 rc 0/0 00000100:00100000:1.0:1789760775.282352:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6137 00000100:00100000:1.0:1789760775.282363:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753664367695 time 753704331745 state 0x2 ref 0 00000001:00080000:2.0:1789760775.283644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973425 is committed 00000100:00100000:0.0:1789760775.290110:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760775.290356:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783168 00000100:00100000:2.0:1789760775.290497:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783168 00000100:00100000:2.0:1789760775.290807:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753664367695 time 753704331745 state 0x0 ref 1 00000100:00100000:2.0:1789760775.290823:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753704592161 time 753704331745 state 0x1 ref 1 00000100:00100000:2.0:1789760775.290859:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753704592161 time 753756147109 state 0x0 ref 1 00000100:00100000:2.0:1789760775.290875:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760775.290930:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25022464, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:2.0:1789760775.290939:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.290954:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760775.290959:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.290966:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.290980:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.321222:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25022464 -> 25026560 00010000:00010000:1.0:1789760775.321242:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.321254:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.321263:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.321271:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.321275:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.321289:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.321309:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.321315:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973426 is committed 00010000:00010000:1.0:1789760775.321316:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d1f975 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.321404:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30527us (31297us total) trans 4294973426 rc 0/0 00000100:00100000:1.0:1789760775.321413:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6138 00000100:00100000:1.0:1789760775.321422:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753704592161 time 753756147109 state 0x2 ref 0 00000100:00100000:0.0:1789760775.325210:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.325258:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783296 00000100:00100000:1.0:1789760775.325269:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783296 00000100:00100000:1.0:1789760775.325322:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753704592161 time 753756147109 state 0x0 ref 1 00000100:00100000:1.0:1789760775.325338:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753756407525 time 753756147109 state 0x1 ref 1 00000100:00100000:1.0:1789760775.325367:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753756407525 time 753790656556 state 0x0 ref 1 00000100:00100000:1.0:1789760775.325380:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.325428:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25026560, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.325437:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.325453:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760775.325457:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.325464:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.325477:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.363749:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973427 is committed 00002000:00010000:0.0:1789760775.363922:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25026560 -> 25030656 00010000:00010000:0.0:1789760775.363938:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.363950:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.363959:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.363966:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.363970:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.363979:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.364003:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.364008:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f97c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.364246:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38860us (39039us total) trans 4294973427 rc 0/0 00000100:00100000:0.0:1789760775.364257:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6139 00000100:00100000:0.0:1789760775.364267:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753756407525 time 753790656556 state 0x2 ref 0 00000100:00100000:2.0:1789760775.367518:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.367579:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783424 00000100:00100000:0.0:1789760775.367587:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783424 00000100:00100000:0.0:1789760775.367627:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753756407525 time 753790656556 state 0x0 ref 1 00000100:00100000:0.0:1789760775.367638:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753790916972 time 753790656556 state 0x1 ref 1 00000100:00100000:0.0:1789760775.367668:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753790916972 time 753832956813 state 0x0 ref 1 00000100:00100000:0.0:1789760775.367679:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760775.367758:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25030656, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:0.0:1789760775.367767:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.367782:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760775.367787:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.367793:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.367807:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.397033:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973428 is committed 00002000:00010000:0.0:1789760775.397144:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25030656 -> 25034752 00010000:00010000:0.0:1789760775.397159:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.397172:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.397180:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.397187:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.397191:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.397200:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.397218:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.397225:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f983 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.397325:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29642us (29810us total) trans 4294973428 rc 0/0 00000100:00100000:0.0:1789760775.397336:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6140 00000100:00100000:0.0:1789760775.397346:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753790916972 time 753832956813 state 0x2 ref 0 00000100:00100000:3.0:1789760775.401265:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.401524:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783552 00000100:00100000:1.0:1789760775.401534:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783552 00000100:00100000:1.0:1789760775.401588:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753790916972 time 753832956813 state 0x0 ref 1 00000100:00100000:1.0:1789760775.401604:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753833217229 time 753832956813 state 0x1 ref 1 00000100:00100000:1.0:1789760775.401637:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753833217229 time 753866926361 state 0x0 ref 1 00000100:00100000:1.0:1789760775.405128:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.405192:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25034752, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.405201:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.405216:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760775.405220:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.405227:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.405240:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.436934:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973429 is committed 00002000:00010000:3.0:1789760775.437212:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25034752 -> 25038848 00010000:00010000:3.0:1789760775.437229:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437243:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437253:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437260:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760775.437265:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437273:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437293:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.437301:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f98a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.437472:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35820us (36212us total) trans 4294973429 rc 0/0 00000100:00100000:3.0:1789760775.437482:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6141 00000100:00100000:3.0:1789760775.437492:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753833217229 time 753866926361 state 0x2 ref 0 00000100:00100000:1.0:1789760775.441187:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760775.441260:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783680 00000100:00100000:2.0:1789760775.441270:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783680 00000100:00100000:2.0:1789760775.441322:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753833217229 time 753866926361 state 0x0 ref 1 00000100:00100000:2.0:1789760775.441334:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753867186777 time 753866926361 state 0x1 ref 1 00000100:00100000:2.0:1789760775.441422:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753867186777 time 753906711325 state 0x0 ref 1 00000100:00100000:2.0:1789760775.441435:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760775.441482:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25038848, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:2.0:1789760775.441491:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.441509:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760775.441513:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.441519:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.441533:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.473077:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973430 is committed 00002000:00010000:0.0:1789760775.473181:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25038848 -> 25042944 00010000:00010000:0.0:1789760775.473197:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473210:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473220:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473228:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.473232:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473242:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473263:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.473275:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1f991 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.473376:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31932us (32187us total) trans 4294973430 rc 0/0 00000100:00100000:0.0:1789760775.473386:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6142 00000100:00100000:0.0:1789760775.473396:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753867186777 time 753906711325 state 0x2 ref 0 00000100:00100000:2.0:1789760775.476901:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.476987:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783808 00000100:00100000:1.0:1789760775.476999:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783808 00000100:00100000:1.0:1789760775.477054:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753867186777 time 753906711325 state 0x0 ref 1 00000100:00100000:1.0:1789760775.477070:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753906971741 time 753906711325 state 0x1 ref 1 00000100:00100000:1.0:1789760775.477129:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753906971741 time 753942418361 state 0x0 ref 1 00000100:00100000:1.0:1789760775.477142:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.477194:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25042944, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.477208:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.477223:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760775.477228:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.477234:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.477248:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.486112:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760775.509179:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973431 is committed 00002000:00010000:0.0:1789760775.509286:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25042944 -> 25047040 00010000:00010000:0.0:1789760775.509302:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.509315:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.509324:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.509332:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.509337:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.509346:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.509371:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.509380:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1f998 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.509468:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515783808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32323us (32572us total) trans 4294973431 rc 0/0 00000100:00100000:0.0:1789760775.509478:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6143 00000100:00100000:0.0:1789760775.509487:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753906971741 time 753942418361 state 0x2 ref 0 00000100:00100000:2.0:1789760775.518853:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.518992:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515783936 00000100:00100000:1.0:1789760775.519002:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515783936 00000100:00100000:1.0:1789760775.519068:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753906971741 time 753942418361 state 0x0 ref 1 00000100:00100000:1.0:1789760775.519084:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753942678777 time 753942418361 state 0x1 ref 1 00000100:00100000:1.0:1789760775.519161:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753942678777 time 753984448271 state 0x0 ref 1 00000100:00100000:1.0:1789760775.519173:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.519219:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25047040, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.519227:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519243:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760775.519248:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.519254:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519269:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.554253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973432 is committed 00002000:00010000:0.0:1789760775.554373:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25047040 -> 25051136 00010000:00010000:0.0:1789760775.554389:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.554414:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.554423:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.554430:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.554435:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.554450:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.554470:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.554478:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1f99f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.554570:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515783936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35393us (35722us total) trans 4294973432 rc 0/0 00000100:00100000:0.0:1789760775.554581:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6144 00000100:00100000:0.0:1789760775.554590:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753942678777 time 753984448271 state 0x2 ref 0 00000100:00100000:1.0:1789760775.571796:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.571855:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784064 00000100:00100000:0.0:1789760775.571865:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784064 00000100:00100000:0.0:1789760775.571917:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753942678777 time 753984448271 state 0x0 ref 1 00000100:00100000:0.0:1789760775.571933:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753984708687 time 753984448271 state 0x1 ref 1 00000100:00100000:0.0:1789760775.571966:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753984708687 time 754037255026 state 0x0 ref 1 00000100:00100000:0.0:1789760775.571978:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760775.572026:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25051136, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:0.0:1789760775.572034:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.572050:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.572054:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.572060:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.572075:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.596891:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973433 is committed 00002000:00010000:0.0:1789760775.597001:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25051136 -> 25055232 00010000:00010000:0.0:1789760775.597015:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.597027:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.597036:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.597047:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.597053:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.597061:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.597079:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.597088:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.597627:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25644us (25833us total) trans 4294973433 rc 0/0 00000100:00100000:0.0:1789760775.597637:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6145 00000100:00100000:0.0:1789760775.597656:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753984708687 time 754037255026 state 0x2 ref 0 00000100:00100000:2.0:1789760775.601408:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.601556:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784192 00000100:00100000:0.0:1789760775.601566:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784192 00000100:00100000:0.0:1789760775.601616:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 753984708687 time 754037255026 state 0x0 ref 1 00000100:00100000:0.0:1789760775.601631:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754037515442 time 754037255026 state 0x1 ref 1 00000100:00100000:0.0:1789760775.601755:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754037515442 time 754066996072 state 0x0 ref 1 00000100:00100000:0.0:1789760775.601770:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515784192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760775.601817:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25055232, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:0.0:1789760775.601826:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.601841:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.601845:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.601852:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.601870:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.638407:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973434 is committed 00002000:00010000:0.0:1789760775.638504:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25055232 -> 25059328 00010000:00010000:0.0:1789760775.638521:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.638534:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.638544:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.638551:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.638556:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.638565:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.638584:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.638592:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1f9ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.638699:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515784192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36927us (37303us total) trans 4294973434 rc 0/0 00000100:00100000:0.0:1789760775.638762:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6146 00000100:00100000:0.0:1789760775.638772:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754037515442 time 754066996072 state 0x2 ref 0 00000100:00100000:2.0:1789760775.643406:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760775.643488:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784320 00000100:00100000:3.0:1789760775.643498:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784320 00000100:00100000:3.0:1789760775.643563:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754037515442 time 754066996072 state 0x0 ref 1 00000100:00100000:3.0:1789760775.643585:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754067256488 time 754066996072 state 0x1 ref 1 00000100:00100000:3.0:1789760775.643620:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754067256488 time 754108908833 state 0x0 ref 1 00000100:00100000:3.0:1789760775.643632:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760775.643677:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25059328, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:3.0:1789760775.643687:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.643702:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.643745:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.643751:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.643765:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.667837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973435 is committed 00002000:00010000:1.0:1789760775.667863:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25059328 -> 25063424 00010000:00010000:1.0:1789760775.667878:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.667891:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.667898:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.667902:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.667906:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.667912:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.667927:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.667935:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1f9b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.668060:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24425us (24658us total) trans 4294973435 rc 0/0 00000100:00100000:1.0:1789760775.668074:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6147 00000100:00100000:1.0:1789760775.668083:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754067256488 time 754108908833 state 0x2 ref 0 00000100:00100000:1.0:1789760775.670410:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760775.670532:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784448 00000100:00100000:2.0:1789760775.670539:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784448 00000100:00100000:2.0:1789760775.670579:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754067256488 time 754108908833 state 0x0 ref 1 00000100:00100000:2.0:1789760775.670591:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754109169249 time 754108908833 state 0x1 ref 1 00000100:00100000:2.0:1789760775.670626:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754109169249 time 754135915675 state 0x0 ref 1 00000100:00100000:2.0:1789760775.670639:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760775.670681:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25063424, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:2.0:1789760775.670690:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.671069:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760775.671076:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.671083:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.671100:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.717173:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973436 is committed 00002000:00010000:3.0:1789760775.717283:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25063424 -> 25067520 00010000:00010000:3.0:1789760775.717299:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717312:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717322:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717339:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760775.717351:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717361:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717380:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.717388:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1f9bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.717476:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46834us (47066us total) trans 4294973436 rc 0/0 00000100:00100000:3.0:1789760775.717487:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6148 00000100:00100000:3.0:1789760775.717496:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754109169249 time 754135915675 state 0x2 ref 0 00000100:00100000:2.0:1789760775.721968:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.722173:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784576 00000100:00100000:1.0:1789760775.722184:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784576 00000100:00100000:1.0:1789760775.722356:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754109169249 time 754135915675 state 0x0 ref 1 00000100:00100000:1.0:1789760775.722372:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754136176091 time 754135915675 state 0x1 ref 1 00000100:00100000:1.0:1789760775.722440:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754136176091 time 754187727913 state 0x0 ref 1 00000100:00100000:1.0:1789760775.722454:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515784576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.722774:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25067520, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.722783:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.722799:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.722803:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.722810:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.722824:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.759666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973437 is committed 00002000:00010000:1.0:1789760775.759855:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25067520 -> 25071616 00010000:00010000:1.0:1789760775.759870:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.759882:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.759891:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.759898:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.759903:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.759912:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.759931:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.759939:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.760078:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515784576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37613us (38107us total) trans 4294973437 rc 0/0 00000100:00100000:1.0:1789760775.760161:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6149 00000100:00100000:1.0:1789760775.760173:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754136176091 time 754187727913 state 0x2 ref 0 00000100:00100000:2.0:1789760775.765801:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.765845:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784704 00000100:00100000:0.0:1789760775.765856:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784704 00000100:00100000:0.0:1789760775.765912:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754136176091 time 754187727913 state 0x0 ref 1 00000100:00100000:0.0:1789760775.765927:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754187988329 time 754187727913 state 0x1 ref 1 00000100:00100000:0.0:1789760775.765957:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754187988329 time 754231246374 state 0x0 ref 1 00000100:00100000:0.0:1789760775.765969:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515784704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760775.766018:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25071616, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:0.0:1789760775.766027:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.766043:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.766047:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.766054:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.766079:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.809061:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973438 is committed 00002000:00010000:2.0:1789760775.809160:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25071616 -> 25075712 00010000:00010000:2.0:1789760775.809184:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.809198:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.809206:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.809213:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760775.809218:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.809232:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.809252:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.809259:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1f9c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.809358:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515784704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43386us (43562us total) trans 4294973438 rc 0/0 00000100:00100000:2.0:1789760775.809376:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6150 00000100:00100000:2.0:1789760775.809387:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754187988329 time 754231246374 state 0x2 ref 0 00000100:00100000:1.0:1789760775.813216:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.814126:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784832 00000100:00100000:0.0:1789760775.814137:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784832 00000100:00100000:1.0:1789760775.814276:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754187988329 time 754231246374 state 0x0 ref 1 00000100:00100000:1.0:1789760775.814294:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754231506790 time 754231246374 state 0x1 ref 1 00000100:00100000:1.0:1789760775.814363:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754231506790 time 754279651923 state 0x0 ref 1 00000100:00100000:1.0:1789760775.814386:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515784832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.814437:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25075712, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xc007 00010000:00010000:1.0:1789760775.814446:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.814460:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.814465:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.814471:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.814484:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.839951:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973439 is committed 00002000:00010000:1.0:1789760775.839977:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25075712 -> 25079808 00002000:00010000:1.0:1789760775.839983:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:1.0:1789760775.839985:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00010000:00010000:1.0:1789760775.839999:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.840016:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.840026:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.840032:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.840037:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.840046:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.840065:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.840072:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.840226:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515784832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25772us (26949us total) trans 4294973439 rc 0/0 00000100:00100000:1.0:1789760775.840238:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6151 00000100:00100000:1.0:1789760775.840248:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754231506790 time 754279651923 state 0x2 ref 0 00000100:00100000:3.0:1789760775.844599:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760775.845620:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515784960 00000100:00100000:0.0:1789760775.845631:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515784960 00000100:00100000:0.0:1789760775.845686:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754231506790 time 754279651923 state 0x0 ref 1 00000100:00100000:0.0:1789760775.845701:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754279912339 time 754279651923 state 0x1 ref 1 00000100:00100000:0.0:1789760775.845765:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754279912339 time 754311054170 state 0x0 ref 1 00000100:00100000:0.0:1789760775.845788:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760775.845836:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25079808, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:0.0:1789760775.845845:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.845860:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.845864:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.845870:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.845883:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.896465:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973440 is committed 00002000:00010000:0.0:1789760775.896513:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25079808 -> 25083904 00010000:00010000:0.0:1789760775.896530:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.896544:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.896553:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.896560:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760775.896565:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.896574:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.896595:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.896603:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1f9d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.897058:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515784960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51276us (52461us total) trans 4294973440 rc 0/0 00000100:00100000:1.0:1789760775.897069:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6152 00000100:00100000:1.0:1789760775.897079:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754279912339 time 754311054170 state 0x2 ref 0 00000100:00100000:2.0:1789760775.902449:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760775.902672:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785088 00000100:00100000:1.0:1789760775.902682:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785088 00000100:00100000:1.0:1789760775.902784:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754279912339 time 754311054170 state 0x0 ref 1 00000100:00100000:1.0:1789760775.902798:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754311314586 time 754311054170 state 0x1 ref 1 00000100:00100000:1.0:1789760775.902827:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754311314586 time 754368116232 state 0x0 ref 1 00000100:00100000:1.0:1789760775.902839:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515785088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760775.902887:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25083904, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760775.902896:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.902913:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.902917:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.902923:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.902935:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.960036:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973441 is committed 00002000:00010000:1.0:1789760775.960233:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25083904 -> 25088000 00010000:00010000:1.0:1789760775.960248:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.960262:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.960270:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.960277:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760775.960281:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.960291:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.960370:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.960378:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.960468:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515785088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57626us (58023us total) trans 4294973441 rc 0/0 00000100:00100000:1.0:1789760775.960477:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6153 00000100:00100000:1.0:1789760775.960487:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754311314586 time 754368116232 state 0x2 ref 0 00000100:00100000:2.0:1789760775.974269:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760775.974352:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785216 00000100:00100000:3.0:1789760775.974363:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785216 00000100:00100000:3.0:1789760775.974418:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754311314586 time 754368116232 state 0x0 ref 1 00000100:00100000:3.0:1789760775.974433:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754368376648 time 754368116232 state 0x1 ref 1 00000100:00100000:3.0:1789760775.974515:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754368376648 time 754439803696 state 0x0 ref 1 00000100:00100000:3.0:1789760775.974527:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515785216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760775.974571:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25088000, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760775.974577:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.974591:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.974594:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.974598:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.974608:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.002953:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973442 is committed 00002000:00010000:1.0:1789760776.003042:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25088000 -> 25092096 00010000:00010000:1.0:1789760776.003062:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.003086:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.003097:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.003105:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.003109:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.003119:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.003138:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.003151:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1f9e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.003336:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515785216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28803us (29070us total) trans 4294973442 rc 0/0 00000100:00100000:1.0:1789760776.003347:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6154 00000100:00100000:1.0:1789760776.003356:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754368376648 time 754439803696 state 0x2 ref 0 00000100:00100000:3.0:1789760776.010639:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.010775:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785344 00000100:00100000:1.0:1789760776.010786:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785344 00000100:00100000:1.0:1789760776.010858:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754368376648 time 754439803696 state 0x0 ref 1 00000100:00100000:1.0:1789760776.010874:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754440064112 time 754439803696 state 0x1 ref 1 00000100:00100000:1.0:1789760776.010911:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754440064112 time 754476200202 state 0x0 ref 1 00000100:00100000:1.0:1789760776.010926:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515785344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.010986:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25092096, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.010998:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.011016:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760776.011021:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.011027:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.011310:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760776.012151:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760776.040258:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973443 is committed 00002000:00010000:1.0:1789760776.040331:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25092096 -> 25096192 00010000:00010000:1.0:1789760776.040346:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.040358:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.040372:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.040378:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.040383:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.040391:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.040411:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.040417:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.040515:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515785344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29588us (29884us total) trans 4294973443 rc 0/0 00000100:00100000:1.0:1789760776.040525:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6155 00000100:00100000:1.0:1789760776.040535:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754440064112 time 754476200202 state 0x2 ref 0 00000100:00100000:2.0:1789760776.043843:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760776.045524:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785472 00000100:00100000:0.0:1789760776.045539:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785472 00000100:00100000:1.0:1789760776.046866:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754440064112 time 754476200202 state 0x0 ref 1 00000100:00100000:1.0:1789760776.046883:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754476460618 time 754476200202 state 0x1 ref 1 00000100:00100000:1.0:1789760776.046920:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754476460618 time 754512209166 state 0x0 ref 1 00000100:00100000:1.0:1789760776.046934:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515785472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.046982:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25096192, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.046992:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.047009:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760776.047014:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.047020:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.047036:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.080027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973444 is committed 00002000:00010000:1.0:1789760776.080876:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25096192 -> 25100288 00010000:00010000:1.0:1789760776.080893:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.080906:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.080915:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.080922:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.080927:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.080935:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.080954:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.080961:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1f9f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.081059:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515785472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34124us (37220us total) trans 4294973444 rc 0/0 00000100:00100000:1.0:1789760776.081069:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6156 00000100:00100000:1.0:1789760776.081152:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754476460618 time 754512209166 state 0x2 ref 0 00000100:00100000:3.0:1789760776.088304:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.088387:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785600 00000100:00100000:1.0:1789760776.088397:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785600 00000100:00100000:1.0:1789760776.088445:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754476460618 time 754512209166 state 0x0 ref 1 00000100:00100000:1.0:1789760776.088462:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754512469582 time 754512209166 state 0x1 ref 1 00000100:00100000:1.0:1789760776.088618:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754512469582 time 754553905261 state 0x0 ref 1 00000100:00100000:1.0:1789760776.088631:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515785600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.088687:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25100288, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.088696:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.088751:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760776.088756:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.088771:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.088786:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.138106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973445 is committed 00002000:00010000:3.0:1789760776.138204:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25100288 -> 25104384 00010000:00010000:3.0:1789760776.138218:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.138232:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.138247:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.138254:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760776.138260:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.138268:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.138286:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.138294:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1f9fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.138384:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515785600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49751us (50084us total) trans 4294973445 rc 0/0 00000100:00100000:3.0:1789760776.138394:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6157 00000100:00100000:3.0:1789760776.138404:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754512469582 time 754553905261 state 0x2 ref 0 00000100:00100000:3.0:1789760776.152283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760776.152471:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785728 00000100:00100000:0.0:1789760776.152482:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785728 00000100:00100000:0.0:1789760776.152540:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754512469582 time 754553905261 state 0x0 ref 1 00000100:00100000:0.0:1789760776.152556:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754554165677 time 754553905261 state 0x1 ref 1 00000100:00100000:0.0:1789760776.152588:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754554165677 time 754617876761 state 0x0 ref 1 00000100:00100000:0.0:1789760776.152600:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515785728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760776.152662:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25104384, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:0.0:1789760776.152672:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.152688:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760776.152693:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.152701:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.152767:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.230113:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973446 is committed 00002000:00010000:0.0:1789760776.230216:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25104384 -> 25108480 00010000:00010000:0.0:1789760776.230231:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.230244:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.230253:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.230260:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760776.230266:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.230275:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.230295:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.230303:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fa01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.230498:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515785728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 77893us (78218us total) trans 4294973446 rc 0/0 00000100:00100000:0.0:1789760776.230510:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6158 00000100:00100000:0.0:1789760776.230519:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754554165677 time 754617876761 state 0x2 ref 0 00000100:00100000:0.0:1789760776.233676:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.233977:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785856 00000100:00100000:3.0:1789760776.233987:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785856 00000100:00100000:3.0:1789760776.234037:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754554165677 time 754617876761 state 0x0 ref 1 00000100:00100000:3.0:1789760776.234052:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754618137177 time 754617876761 state 0x1 ref 1 00000100:00100000:3.0:1789760776.234115:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754618137177 time 754699404287 state 0x0 ref 1 00000100:00100000:3.0:1789760776.234129:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515785856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.234176:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25108480, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.234184:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.234200:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760776.234204:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.234210:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.234223:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.266928:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25108480 -> 25112576 00010000:00010000:0.0:1789760776.266948:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.266962:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.266971:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.266984:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760776.266989:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.266998:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.267017:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.267025:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fa08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.267115:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515785856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32983us (33443us total) trans 4294973447 rc 0/0 00000100:00100000:0.0:1789760776.267125:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6159 00000100:00100000:0.0:1789760776.267134:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754618137177 time 754699404287 state 0x2 ref 0 00000001:00080000:2.0:1789760776.269249:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973447 is committed 00000100:00100000:3.0:1789760776.273528:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.273664:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515785984 00000100:00100000:1.0:1789760776.273675:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515785984 00000100:00100000:1.0:1789760776.273765:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754618137177 time 754699404287 state 0x0 ref 1 00000100:00100000:1.0:1789760776.273851:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754699664703 time 754699404287 state 0x1 ref 1 00000100:00100000:1.0:1789760776.273924:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754699664703 time 754739212356 state 0x0 ref 1 00000100:00100000:1.0:1789760776.273937:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515785984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.273991:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25112576, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.274004:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274021:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760776.274026:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.274032:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274065:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.310660:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25112576 -> 25116672 00010000:00010000:2.0:1789760776.310682:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310695:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.310786:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973448 is committed 00010000:00010000:2.0:1789760776.310943:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310954:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760776.310960:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310970:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310990:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.310998:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fa0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.311089:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515785984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37150us (37566us total) trans 4294973448 rc 0/0 00000100:00100000:2.0:1789760776.311100:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6160 00000100:00100000:2.0:1789760776.311110:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754699664703 time 754739212356 state 0x2 ref 0 00000100:00100000:3.0:1789760776.314754:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.314985:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786112 00000100:00100000:1.0:1789760776.314996:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786112 00000100:00100000:1.0:1789760776.315142:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754699664703 time 754739212356 state 0x0 ref 1 00000100:00100000:1.0:1789760776.315156:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754739472772 time 754739212356 state 0x1 ref 1 00000100:00100000:1.0:1789760776.315231:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754739472772 time 754780519075 state 0x0 ref 1 00000100:00100000:1.0:1789760776.315243:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.315293:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25116672, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.315303:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.315317:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760776.315321:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.315327:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.315339:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.363347:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973449 is committed 00002000:00010000:0.0:1789760776.363560:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25116672 -> 25120768 00010000:00010000:0.0:1789760776.363576:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.363590:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.363605:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.363613:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760776.363618:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.363626:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.363658:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.363666:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.363800:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48554us (49050us total) trans 4294973449 rc 0/0 00000100:00100000:0.0:1789760776.363811:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6161 00000100:00100000:0.0:1789760776.363820:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754739472772 time 754780519075 state 0x2 ref 0 00000100:00100000:3.0:1789760776.367268:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.367359:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786240 00000100:00100000:1.0:1789760776.367375:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786240 00000100:00100000:1.0:1789760776.367428:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754739472772 time 754780519075 state 0x0 ref 1 00000100:00100000:1.0:1789760776.367445:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754780779491 time 754780519075 state 0x1 ref 1 00000100:00100000:1.0:1789760776.367475:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754780779491 time 754832764022 state 0x0 ref 1 00000100:00100000:1.0:1789760776.367487:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.367546:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25120768, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.367554:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.367568:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760776.367572:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.367577:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.367590:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.419086:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973450 is committed 00002000:00010000:3.0:1789760776.419250:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25120768 -> 25124864 00010000:00010000:3.0:1789760776.419266:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419285:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419295:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419302:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760776.419307:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419317:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419336:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.419344:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.419445:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51955us (52180us total) trans 4294973450 rc 0/0 00000100:00100000:3.0:1789760776.419456:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6162 00000100:00100000:3.0:1789760776.419466:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754780779491 time 754832764022 state 0x2 ref 0 00000100:00100000:1.0:1789760776.435845:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.436040:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786368 00000100:00100000:3.0:1789760776.436052:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786368 00000100:00100000:3.0:1789760776.436301:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754780779491 time 754832764022 state 0x0 ref 1 00000100:00100000:3.0:1789760776.436317:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754833024438 time 754832764022 state 0x1 ref 1 00000100:00100000:3.0:1789760776.436351:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754833024438 time 754901639883 state 0x0 ref 1 00000100:00100000:3.0:1789760776.436365:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.436416:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25124864, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.436425:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.436440:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760776.436444:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.436450:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.436464:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.471454:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973451 is committed 00002000:00010000:3.0:1789760776.471647:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25124864 -> 25128960 00010000:00010000:3.0:1789760776.471665:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.471678:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.471692:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.471699:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760776.471703:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.471751:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.471769:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.471776:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.471854:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35489us (36014us total) trans 4294973451 rc 0/0 00000100:00100000:3.0:1789760776.471863:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6163 00000100:00100000:3.0:1789760776.471872:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754833024438 time 754901639883 state 0x2 ref 0 00000100:00100000:2.0:1789760776.477942:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760776.478093:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786496 00000100:00100000:0.0:1789760776.478103:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786496 00000100:00100000:0.0:1789760776.478158:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754833024438 time 754901639883 state 0x0 ref 1 00000100:00100000:0.0:1789760776.478176:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754901900299 time 754901639883 state 0x1 ref 1 00000100:00100000:0.0:1789760776.478205:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754901900299 time 754943494097 state 0x0 ref 1 00000100:00100000:0.0:1789760776.478218:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760776.478267:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25128960, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:0.0:1789760776.478277:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.478293:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760776.478298:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.478305:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.478319:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.510297:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760776.520222:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25128960 -> 25133056 00010000:00010000:2.0:1789760776.520241:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520254:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520268:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520276:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760776.520281:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520290:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520308:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520316:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fa2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.520409:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515786496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42189us (42471us total) trans 4294973452 rc 0/0 00000100:00100000:2.0:1789760776.520420:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6164 00000100:00100000:2.0:1789760776.520429:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754901900299 time 754943494097 state 0x2 ref 0 00000001:00080000:3.0:1789760776.520987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973452 is committed 00000100:00100000:0.0:1789760776.523371:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.525623:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786624 00000100:00100000:3.0:1789760776.525635:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786624 00000100:00100000:3.0:1789760776.525850:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754901900299 time 754943494097 state 0x0 ref 1 00000100:00100000:3.0:1789760776.525870:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754943754513 time 754943494097 state 0x1 ref 1 00000100:00100000:3.0:1789760776.525955:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754943754513 time 754991242446 state 0x0 ref 1 00000100:00100000:3.0:1789760776.525969:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515786624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.526031:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25133056, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.526040:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.526056:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760776.526060:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.526067:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.526081:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.551756:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973453 is committed 00002000:00010000:0.0:1789760776.551865:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25133056 -> 25137152 00010000:00010000:0.0:1789760776.551878:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.551888:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.551898:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.551902:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760776.551906:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.551911:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.551928:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.551932:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fa32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.552043:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515786624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26072us (28675us total) trans 4294973453 rc 0/0 00000100:00100000:0.0:1789760776.552054:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6165 00000100:00100000:0.0:1789760776.552064:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754943754513 time 754991242446 state 0x2 ref 0 00000100:00100000:0.0:1789760776.556949:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760776.557057:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786752 00000100:00100000:2.0:1789760776.557068:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786752 00000100:00100000:2.0:1789760776.557266:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754943754513 time 754991242446 state 0x0 ref 1 00000100:00100000:2.0:1789760776.557283:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754991502862 time 754991242446 state 0x1 ref 1 00000100:00100000:2.0:1789760776.557317:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754991502862 time 755022605969 state 0x0 ref 1 00000100:00100000:2.0:1789760776.557330:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515786752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760776.557386:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25137152, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:2.0:1789760776.557395:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.557410:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.557415:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.557422:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.557436:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.597267:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973454 is committed 00002000:00010000:2.0:1789760776.597347:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25137152 -> 25141248 00010000:00010000:2.0:1789760776.597431:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.597446:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.597455:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.597462:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760776.597467:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.597475:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.597494:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.597501:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d1fa39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.597633:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515786752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40301us (40688us total) trans 4294973454 rc 0/0 00000100:00100000:2.0:1789760776.597756:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6166 00000100:00100000:2.0:1789760776.597766:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754991502862 time 755022605969 state 0x2 ref 0 00000100:00100000:2.0:1789760776.600971:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.601118:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515786880 00000100:00100000:1.0:1789760776.601127:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515786880 00000100:00100000:1.0:1789760776.601178:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 754991502862 time 755022605969 state 0x0 ref 1 00000100:00100000:1.0:1789760776.601192:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755022866385 time 755022605969 state 0x1 ref 1 00000100:00100000:1.0:1789760776.601223:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755022866385 time 755066512460 state 0x0 ref 1 00000100:00100000:1.0:1789760776.601236:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515786880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.601286:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25141248, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.601295:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.601311:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.601315:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.601321:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.601335:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760776.629750:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25141248 -> 25145344 00010000:00010000:1.0:1789760776.629771:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.629784:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.629793:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.629801:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.629806:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.629815:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.629835:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.629843:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.629921:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515786880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28684us (28955us total) trans 4294973455 rc 0/0 00000100:00100000:1.0:1789760776.629935:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6167 00000100:00100000:1.0:1789760776.629944:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755022866385 time 755066512460 state 0x2 ref 0 00000001:00080000:1.0:1789760776.629978:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973455 is committed 00000100:00100000:0.0:1789760776.633680:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.634445:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787008 00000100:00100000:1.0:1789760776.634456:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787008 00000100:00100000:1.0:1789760776.634509:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755022866385 time 755066512460 state 0x0 ref 1 00000100:00100000:1.0:1789760776.634523:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755066772876 time 755066512460 state 0x1 ref 1 00000100:00100000:1.0:1789760776.634654:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755066772876 time 755099942328 state 0x0 ref 1 00000100:00100000:1.0:1789760776.634667:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.634762:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25145344, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.634772:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.634787:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.634802:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.634809:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.634823:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.664883:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973456 is committed 00002000:00010000:1.0:1789760776.664975:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25145344 -> 25149440 00010000:00010000:1.0:1789760776.664990:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.665003:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.665069:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.665078:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.665084:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.665094:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.665115:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.665122:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fa47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.665210:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30540us (31534us total) trans 4294973456 rc 0/0 00000100:00100000:1.0:1789760776.665221:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6168 00000100:00100000:1.0:1789760776.665231:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755066772876 time 755099942328 state 0x2 ref 0 00000100:00100000:2.0:1789760776.668827:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.668923:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787136 00000100:00100000:3.0:1789760776.668934:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787136 00000100:00100000:3.0:1789760776.668986:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755066772876 time 755099942328 state 0x0 ref 1 00000100:00100000:3.0:1789760776.669071:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755100202744 time 755099942328 state 0x1 ref 1 00000100:00100000:3.0:1789760776.669127:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755100202744 time 755134416623 state 0x0 ref 1 00000100:00100000:3.0:1789760776.669139:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515787136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.669258:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25149440, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.669267:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.669284:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.669288:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.669295:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.669307:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.698991:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973457 is committed 00002000:00010000:3.0:1789760776.699154:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25149440 -> 25153536 00010000:00010000:3.0:1789760776.699169:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.699183:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.699192:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.699199:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760776.699205:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.699214:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.699233:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.699240:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.699360:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515787136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30218us (30538us total) trans 4294973457 rc 0/0 00000100:00100000:3.0:1789760776.699371:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6169 00000100:00100000:3.0:1789760776.699380:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755100202744 time 755134416623 state 0x2 ref 0 00000100:00100000:2.0:1789760776.704047:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.704162:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787264 00000100:00100000:3.0:1789760776.704173:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787264 00000100:00100000:3.0:1789760776.704224:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755100202744 time 755134416623 state 0x0 ref 1 00000100:00100000:3.0:1789760776.704238:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755134677039 time 755134416623 state 0x1 ref 1 00000100:00100000:3.0:1789760776.704268:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755134677039 time 755169557062 state 0x0 ref 1 00000100:00100000:3.0:1789760776.704280:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.704326:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25153536, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.704335:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.704349:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.704354:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.704361:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.704374:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.735563:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25153536 -> 25157632 00010000:00010000:0.0:1789760776.735594:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.735607:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.735615:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.735622:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760776.735627:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.735649:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.735669:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.735677:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fa55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.735800:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515787264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31517us (31755us total) trans 4294973458 rc 0/0 00000100:00100000:0.0:1789760776.735811:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6170 00000100:00100000:0.0:1789760776.735820:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755134677039 time 755169557062 state 0x2 ref 0 00000001:00080000:1.0:1789760776.736780:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973458 is committed 00000100:00100000:0.0:1789760776.742221:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.742414:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787392 00000100:00100000:3.0:1789760776.742423:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787392 00000100:00100000:3.0:1789760776.742475:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755134677039 time 755169557062 state 0x0 ref 1 00000100:00100000:3.0:1789760776.742755:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755169817478 time 755169557062 state 0x1 ref 1 00000100:00100000:3.0:1789760776.742788:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755169817478 time 755208077413 state 0x0 ref 1 00000100:00100000:3.0:1789760776.742801:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515787392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.742847:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25157632, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.742855:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.742870:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.742874:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.742880:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.742893:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.773427:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973459 is committed 00002000:00010000:1.0:1789760776.773536:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25157632 -> 25161728 00010000:00010000:1.0:1789760776.773552:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.773565:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.773573:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.773580:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.773585:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.773595:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.773614:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.773621:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fa5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.774034:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515787392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31228us (31815us total) trans 4294973459 rc 0/0 00000100:00100000:1.0:1789760776.774054:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6171 00000100:00100000:1.0:1789760776.774068:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755169817478 time 755208077413 state 0x2 ref 0 00000100:00100000:2.0:1789760776.778555:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.778644:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787520 00000100:00100000:3.0:1789760776.778653:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787520 00000100:00100000:3.0:1789760776.778701:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755169817478 time 755208077413 state 0x0 ref 1 00000100:00100000:3.0:1789760776.778755:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755208337829 time 755208077413 state 0x1 ref 1 00000100:00100000:3.0:1789760776.778788:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755208337829 time 755244068966 state 0x0 ref 1 00000100:00100000:3.0:1789760776.778801:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.778853:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25161728, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:3.0:1789760776.778861:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.778877:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.778881:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.778888:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.778901:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.807247:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973460 is committed 00002000:00010000:2.0:1789760776.807313:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25161728 -> 25165824 00010000:00010000:2.0:1789760776.807328:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.807347:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.807356:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.807374:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760776.807379:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.807389:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.807407:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.807414:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fa63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.807504:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28700us (28953us total) trans 4294973460 rc 0/0 00000100:00100000:2.0:1789760776.807514:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6172 00000100:00100000:2.0:1789760776.807523:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755208337829 time 755244068966 state 0x2 ref 0 00000100:00100000:2.0:1789760776.811044:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.811143:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787648 00000100:00100000:1.0:1789760776.811153:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787648 00000100:00100000:1.0:1789760776.811201:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755208337829 time 755244068966 state 0x0 ref 1 00000100:00100000:1.0:1789760776.811217:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755244329382 time 755244068966 state 0x1 ref 1 00000100:00100000:1.0:1789760776.811248:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755244329382 time 755276537047 state 0x0 ref 1 00000100:00100000:1.0:1789760776.811260:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.811306:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25165824, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xc007 00010000:00010000:1.0:1789760776.811315:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.811330:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.811334:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.811341:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.811354:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.837019:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973461 is committed 00002000:00010000:1.0:1789760776.837146:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25165824 -> 25169920 00002000:00010000:1.0:1789760776.837151:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:1.0:1789760776.837154:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00002000:00010000:1.0:1789760776.837157:0:13070:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:1.0:1789760776.837170:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.837183:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.837193:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.837201:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.837206:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.837214:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.837233:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.837241:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.837326:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26064us (26288us total) trans 4294973461 rc 0/0 00000100:00100000:1.0:1789760776.837336:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6173 00000100:00100000:1.0:1789760776.837346:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755244329382 time 755276537047 state 0x2 ref 0 00000100:00100000:1.0:1789760776.841564:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.841652:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787776 00000100:00100000:3.0:1789760776.841662:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787776 00000100:00100000:3.0:1789760776.841743:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755244329382 time 755276537047 state 0x0 ref 1 00000100:00100000:3.0:1789760776.841759:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755276797463 time 755276537047 state 0x1 ref 1 00000100:00100000:1.0:1789760776.841933:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755276797463 time 755307220590 state 0x0 ref 1 00000100:00100000:1.0:1789760776.841947:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.841994:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25169920, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760776.842003:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.842017:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.842022:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.842028:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.842042:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.875539:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25169920 -> 25174016 00010000:00010000:2.0:1789760776.875557:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.875567:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.875573:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.875578:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00000001:00080000:0.0:1789760776.875645:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973462 is committed 00010000:00010000:2.0:1789760776.875685:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.875697:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.875749:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.875758:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fa71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.875884:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33934us (34323us total) trans 4294973462 rc 0/0 00000100:00100000:2.0:1789760776.875894:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6174 00000100:00100000:2.0:1789760776.875904:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755276797463 time 755307220590 state 0x2 ref 0 00000100:00100000:2.0:1789760776.878559:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760776.878786:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515787904 00000100:00100000:1.0:1789760776.878797:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515787904 00000100:00100000:1.0:1789760776.878850:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755276797463 time 755307220590 state 0x0 ref 1 00000100:00100000:1.0:1789760776.878865:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755307481006 time 755307220590 state 0x1 ref 1 00000100:00100000:1.0:1789760776.878896:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755307481006 time 755344185388 state 0x0 ref 1 00000100:00100000:1.0:1789760776.878909:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515787904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760776.878963:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25174016, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760776.878974:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.878995:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.878999:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.879006:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.879025:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.911175:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25174016 -> 25178112 00010000:00010000:2.0:1789760776.911195:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.911208:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.911217:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.911224:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760776.911229:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.911238:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.911257:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.911264:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fa78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.911358:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515787904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32447us (32804us total) trans 4294973463 rc 0/0 00000100:00100000:2.0:1789760776.911775:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6175 00000100:00100000:2.0:1789760776.911788:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755307481006 time 755344185388 state 0x2 ref 0 00000001:00080000:0.0:1789760776.911916:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973463 is committed 00000100:00100000:0.0:1789760776.917650:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.917756:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788032 00000100:00100000:3.0:1789760776.917766:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788032 00000100:00100000:3.0:1789760776.917819:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755307481006 time 755344185388 state 0x0 ref 1 00000100:00100000:3.0:1789760776.917834:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755344445804 time 755344185388 state 0x1 ref 1 00000100:00100000:3.0:1789760776.917869:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755344445804 time 755383157526 state 0x0 ref 1 00000100:00100000:3.0:1789760776.917881:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.917926:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25178112, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:3.0:1789760776.917935:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.917951:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.917955:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.917962:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.917976:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.946548:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973464 is committed 00002000:00010000:0.0:1789760776.946632:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25178112 -> 25182208 00010000:00010000:0.0:1789760776.946654:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.946666:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.946675:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.946683:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760776.946688:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.946697:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.946835:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.946845:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fa7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.946945:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29054us (29292us total) trans 4294973464 rc 0/0 00000100:00100000:0.0:1789760776.946954:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6176 00000100:00100000:0.0:1789760776.946965:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755344445804 time 755383157526 state 0x2 ref 0 00000100:00100000:1.0:1789760776.953480:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.953604:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788160 00000100:00100000:3.0:1789760776.953615:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788160 00000100:00100000:3.0:1789760776.953682:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755344445804 time 755383157526 state 0x0 ref 1 00000100:00100000:3.0:1789760776.953697:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755383417942 time 755383157526 state 0x1 ref 1 00000100:00100000:3.0:1789760776.953807:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755383417942 time 755419096255 state 0x0 ref 1 00000100:00100000:3.0:1789760776.953820:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760776.953866:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25182208, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:3.0:1789760776.953875:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.953891:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.953895:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.953901:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.953915:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.993003:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973465 is committed 00002000:00010000:1.0:1789760776.993225:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25182208 -> 25186304 00010000:00010000:1.0:1789760776.993242:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.993257:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.993266:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.993272:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760776.993278:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.993291:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.993312:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.993319:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fa86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.993483:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39660us (40009us total) trans 4294973465 rc 0/0 00000100:00100000:1.0:1789760776.993494:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6177 00000100:00100000:1.0:1789760776.993503:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755383417942 time 755419096255 state 0x2 ref 0 00000100:00100000:2.0:1789760776.998272:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760776.998479:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788288 00000100:00100000:3.0:1789760776.998489:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788288 00000100:00100000:2.0:1789760776.998850:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755383417942 time 755419096255 state 0x0 ref 1 00000100:00100000:2.0:1789760776.998870:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755419356671 time 755419096255 state 0x1 ref 1 00000100:00100000:2.0:1789760776.998931:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755419356671 time 755464219623 state 0x0 ref 1 00000100:00100000:2.0:1789760776.998945:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760776.998992:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25186304, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760776.999001:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.999017:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.999021:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.999028:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.999043:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.039979:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973466 is committed 00002000:00010000:0.0:1789760777.040074:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25186304 -> 25190400 00010000:00010000:0.0:1789760777.040090:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.040103:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.040111:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.040119:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760777.040124:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.040133:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.040153:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.040166:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fa8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.040293:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41346us (42027us total) trans 4294973466 rc 0/0 00000100:00100000:0.0:1789760777.040304:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6178 00000100:00100000:0.0:1789760777.040314:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755419356671 time 755464219623 state 0x2 ref 0 00000100:00100000:0.0:1789760777.044005:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760777.044092:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788416 00000100:00100000:1.0:1789760777.044102:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788416 00000100:00100000:1.0:1789760777.044151:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755419356671 time 755464219623 state 0x0 ref 1 00000100:00100000:1.0:1789760777.044167:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755464480039 time 755464219623 state 0x1 ref 1 00000100:00100000:1.0:1789760777.044224:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755464480039 time 755509512897 state 0x0 ref 1 00000100:00100000:1.0:1789760777.044236:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760777.044284:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25190400, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760777.044294:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.044309:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760777.044314:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.044320:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.044333:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.072453:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973467 is committed 00002000:00010000:2.0:1789760777.072587:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25190400 -> 25194496 00010000:00010000:2.0:1789760777.072602:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.072615:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.072624:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.072631:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.072644:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.072654:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.072673:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.072680:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fa94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.072813:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28572us (28810us total) trans 4294973467 rc 0/0 00000100:00100000:2.0:1789760777.072825:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6179 00000100:00100000:2.0:1789760777.072836:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755464480039 time 755509512897 state 0x2 ref 0 00000100:00100000:1.0:1789760777.076663:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.076786:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788544 00000100:00100000:2.0:1789760777.076795:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788544 00000100:00100000:2.0:1789760777.076851:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755464480039 time 755509512897 state 0x0 ref 1 00000100:00100000:2.0:1789760777.076866:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755509773313 time 755509512897 state 0x1 ref 1 00000100:00100000:2.0:1789760777.076897:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755509773313 time 755542185936 state 0x0 ref 1 00000100:00100000:2.0:1789760777.076910:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.076954:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25194496, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.076963:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.076979:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760777.076984:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.076990:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.077003:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.102287:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973468 is committed 00002000:00010000:2.0:1789760777.102427:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25194496 -> 25198592 00010000:00010000:2.0:1789760777.102445:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.102459:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.102473:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.102480:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.102485:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.102494:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.102514:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.102521:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fa9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.102623:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25711us (25966us total) trans 4294973468 rc 0/0 00000100:00100000:2.0:1789760777.102633:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6180 00000100:00100000:2.0:1789760777.102642:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755509773313 time 755542185936 state 0x2 ref 0 00000100:00100000:1.0:1789760777.105592:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.105650:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788672 00000100:00100000:2.0:1789760777.105661:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788672 00000100:00100000:2.0:1789760777.105757:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755509773313 time 755542185936 state 0x0 ref 1 00000100:00100000:2.0:1789760777.105772:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755542446352 time 755542185936 state 0x1 ref 1 00000100:00100000:2.0:1789760777.105794:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755542446352 time 755571083760 state 0x0 ref 1 00000100:00100000:2.0:1789760777.105807:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937cab850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515788672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.105853:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25198592, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.105861:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.105877:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760777.105881:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.105888:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.105902:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.132579:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973469 is committed 00002000:00010000:2.0:1789760777.132800:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25198592 -> 25202688 00010000:00010000:2.0:1789760777.132815:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132827:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132836:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132844:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.132850:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132859:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132877:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132884:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1faa2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.132978:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937cab850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515788672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27168us (27387us total) trans 4294973469 rc 0/0 00000100:00100000:2.0:1789760777.132988:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6181 00000100:00100000:2.0:1789760777.132999:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755542446352 time 755571083760 state 0x2 ref 0 00000100:00100000:1.0:1789760777.145938:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760777.146043:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788800 00000100:00100000:0.0:1789760777.146053:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788800 00000100:00100000:0.0:1789760777.146106:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755542446352 time 755571083760 state 0x0 ref 1 00000100:00100000:0.0:1789760777.146121:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755571344176 time 755571083760 state 0x1 ref 1 00000100:00100000:0.0:1789760777.146198:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755571344176 time 755611487665 state 0x0 ref 1 00000100:00100000:0.0:1789760777.146210:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515788800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760777.146260:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25202688, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:0.0:1789760777.146269:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.146285:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760777.146300:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.146307:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.146321:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.193887:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973470 is committed 00002000:00010000:2.0:1789760777.194001:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25202688 -> 25206784 00010000:00010000:2.0:1789760777.194017:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.194030:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.194039:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.194052:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.194056:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.194065:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.194084:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.194091:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1faa9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.194196:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515788800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47982us (48261us total) trans 4294973470 rc 0/0 00000100:00100000:2.0:1789760777.194208:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6182 00000100:00100000:2.0:1789760777.194217:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755571344176 time 755611487665 state 0x2 ref 0 00000100:00100000:0.0:1789760777.198080:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760777.198566:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515788928 00000100:00100000:1.0:1789760777.198576:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515788928 00000100:00100000:1.0:1789760777.198858:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755571344176 time 755611487665 state 0x0 ref 1 00000100:00100000:1.0:1789760777.198874:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755611748081 time 755611487665 state 0x1 ref 1 00000100:00100000:1.0:1789760777.198899:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755611748081 time 755664188981 state 0x0 ref 1 00000100:00100000:1.0:1789760777.198912:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515788928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760777.198964:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25206784, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760777.198973:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.198990:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760777.198994:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.199002:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.199016:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760777.271486:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25206784 -> 25210880 00010000:00010000:1.0:1789760777.271506:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.271519:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.271528:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.271535:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760777.271540:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.271550:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.271570:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.271578:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fab0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.271672:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515788928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 72758us (73597us total) trans 4294973471 rc 0/0 00000100:00100000:1.0:1789760777.271683:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6183 00000100:00100000:1.0:1789760777.271694:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755611748081 time 755664188981 state 0x2 ref 0 00000001:00080000:1.0:1789760777.271801:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973471 is committed 00000100:00100000:1.0:1789760777.274902:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760777.276920:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789056 00000100:00100000:0.0:1789760777.277003:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789056 00000100:00100000:1.0:1789760777.277775:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755611748081 time 755664188981 state 0x0 ref 1 00000100:00100000:1.0:1789760777.277791:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755664449397 time 755664188981 state 0x1 ref 1 00000100:00100000:1.0:1789760777.277817:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755664449397 time 755743106206 state 0x0 ref 1 00000100:00100000:1.0:1789760777.277831:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515789056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760777.277881:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25210880, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760777.277890:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.277907:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760777.277912:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.277918:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.277938:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.307206:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973472 is committed 00002000:00010000:3.0:1789760777.307344:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25210880 -> 25214976 00010000:00010000:3.0:1789760777.307359:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.307372:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.307387:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.307395:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.307400:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.307410:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.307429:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.307437:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fab7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.307597:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515789056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29763us (32700us total) trans 4294973472 rc 0/0 00000100:00100000:3.0:1789760777.307607:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6184 00000100:00100000:3.0:1789760777.307616:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755664449397 time 755743106206 state 0x2 ref 0 00000100:00100000:0.0:1789760777.310840:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.310918:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789184 00000100:00100000:2.0:1789760777.310928:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789184 00000100:00100000:2.0:1789760777.310979:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755664449397 time 755743106206 state 0x0 ref 1 00000100:00100000:2.0:1789760777.310994:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755743366622 time 755743106206 state 0x1 ref 1 00000100:00100000:2.0:1789760777.311027:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755743366622 time 755776316346 state 0x0 ref 1 00000100:00100000:2.0:1789760777.311040:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515789184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.311086:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25214976, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.311095:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.311111:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760777.311115:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.311122:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.311134:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.335813:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973473 is committed 00002000:00010000:3.0:1789760777.335924:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25214976 -> 25219072 00010000:00010000:3.0:1789760777.335942:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.335960:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.335973:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.335981:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.335986:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.335995:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336015:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.336023:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fabe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.336118:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515789184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25075us (25280us total) trans 4294973473 rc 0/0 00000100:00100000:3.0:1789760777.336128:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6185 00000100:00100000:3.0:1789760777.336138:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755743366622 time 755776316346 state 0x2 ref 0 00000100:00100000:1.0:1789760777.340049:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.340146:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789312 00000100:00100000:2.0:1789760777.340156:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789312 00000100:00100000:2.0:1789760777.340216:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755743366622 time 755776316346 state 0x0 ref 1 00000100:00100000:2.0:1789760777.340230:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755776576762 time 755776316346 state 0x1 ref 1 00000100:00100000:2.0:1789760777.340259:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755776576762 time 755805548622 state 0x0 ref 1 00000100:00100000:2.0:1789760777.340271:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515789312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.340318:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25219072, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.340326:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.340342:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760777.340347:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.340353:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.340379:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.370769:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973474 is committed 00002000:00010000:2.0:1789760777.370869:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25219072 -> 25223168 00010000:00010000:2.0:1789760777.370885:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.370897:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.370906:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.370913:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.370918:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.370927:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.370945:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.370953:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fac5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.371039:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515789312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30765us (30995us total) trans 4294973474 rc 0/0 00000100:00100000:2.0:1789760777.371049:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6186 00000100:00100000:2.0:1789760777.371058:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755776576762 time 755805548622 state 0x2 ref 0 00000100:00100000:2.0:1789760777.377317:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760777.377368:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789440 00000100:00100000:1.0:1789760777.377378:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789440 00000100:00100000:1.0:1789760777.377434:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755776576762 time 755805548622 state 0x0 ref 1 00000100:00100000:1.0:1789760777.377451:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755805809038 time 755805548622 state 0x1 ref 1 00000100:00100000:1.0:1789760777.377484:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755805809038 time 755842773388 state 0x0 ref 1 00000100:00100000:1.0:1789760777.377497:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515789440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760777.377547:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25223168, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760777.377556:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.377573:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760777.377577:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.377583:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.377597:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.409822:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973475 is committed 00002000:00010000:2.0:1789760777.409917:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25223168 -> 25227264 00010000:00010000:2.0:1789760777.409933:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.409946:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.409955:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.409963:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.409968:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.409977:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.410002:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.410011:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1facc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.410102:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515789440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32601us (32787us total) trans 4294973475 rc 0/0 00000100:00100000:2.0:1789760777.410112:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6187 00000100:00100000:2.0:1789760777.410122:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755805809038 time 755842773388 state 0x2 ref 0 00000100:00100000:0.0:1789760777.415013:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.415092:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789568 00000100:00100000:2.0:1789760777.415102:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789568 00000100:00100000:2.0:1789760777.415163:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755805809038 time 755842773388 state 0x0 ref 1 00000100:00100000:2.0:1789760777.415188:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755843033804 time 755842773388 state 0x1 ref 1 00000100:00100000:2.0:1789760777.415216:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755843033804 time 755880505179 state 0x0 ref 1 00000100:00100000:2.0:1789760777.415229:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515789568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.415275:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25227264, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.415284:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415300:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760777.415305:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.415311:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415325:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.444941:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973476 is committed 00002000:00010000:2.0:1789760777.445026:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25227264 -> 25231360 00010000:00010000:2.0:1789760777.445041:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.445054:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.445064:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.445071:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.445076:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.445085:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.445102:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.445111:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fad3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.445196:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515789568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29964us (30188us total) trans 4294973476 rc 0/0 00000100:00100000:2.0:1789760777.445205:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6188 00000100:00100000:2.0:1789760777.445215:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755843033804 time 755880505179 state 0x2 ref 0 00000100:00100000:0.0:1789760777.450021:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760777.450627:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789696 00000100:00100000:1.0:1789760777.450702:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789696 00000100:00100000:1.0:1789760777.450832:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755843033804 time 755880505179 state 0x0 ref 1 00000100:00100000:1.0:1789760777.450847:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755880765595 time 755880505179 state 0x1 ref 1 00000100:00100000:1.0:1789760777.450911:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755880765595 time 755916199704 state 0x0 ref 1 00000100:00100000:1.0:1789760777.450925:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515789696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760777.450976:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25231360, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760777.450985:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.451002:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760777.451006:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.451012:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.451026:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.479341:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973477 is committed 00002000:00010000:2.0:1789760777.479437:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25231360 -> 25235456 00010000:00010000:2.0:1789760777.479452:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479466:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479476:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479484:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.479488:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479498:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479519:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479534:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fada lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.479627:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515789696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28699us (29610us total) trans 4294973477 rc 0/0 00000100:00100000:2.0:1789760777.480005:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6189 00000100:00100000:2.0:1789760777.480015:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755880765595 time 755916199704 state 0x2 ref 0 00000100:00100000:1.0:1789760777.485070:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760777.485160:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789824 00000100:00100000:3.0:1789760777.485171:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789824 00000100:00100000:3.0:1789760777.485235:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755880765595 time 755916199704 state 0x0 ref 1 00000100:00100000:3.0:1789760777.485250:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755916460120 time 755916199704 state 0x1 ref 1 00000100:00100000:3.0:1789760777.485324:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755916460120 time 755950612956 state 0x0 ref 1 00000100:00100000:3.0:1789760777.485340:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515789824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760777.485455:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25235456, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:3.0:1789760777.485465:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.485481:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760777.485485:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.485492:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.485505:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.512231:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973478 is committed 00002000:00010000:0.0:1789760777.512331:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25235456 -> 25239552 00010000:00010000:0.0:1789760777.512347:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512361:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512370:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512377:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760777.512383:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512393:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512415:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.512422:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fae1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.512556:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515789824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27214us (27502us total) trans 4294973478 rc 0/0 00000100:00100000:0.0:1789760777.512567:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6190 00000100:00100000:0.0:1789760777.512577:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755916460120 time 755950612956 state 0x2 ref 0 00000100:00100000:0.0:1789760777.517353:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.517766:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515789952 00000100:00100000:2.0:1789760777.517777:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515789952 00000100:00100000:3.0:1789760777.517909:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755916460120 time 755950612956 state 0x0 ref 1 00000100:00100000:3.0:1789760777.517925:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755950873372 time 755950612956 state 0x1 ref 1 00000100:00100000:3.0:1789760777.517989:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755950873372 time 755983277889 state 0x0 ref 1 00000100:00100000:3.0:1789760777.518002:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515789952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760777.518055:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25239552, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:3.0:1789760777.518065:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.518081:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760777.518086:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.518092:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.518106:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.534048:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760777.543087:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973479 is committed 00002000:00010000:3.0:1789760777.543190:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25239552 -> 25243648 00010000:00010000:3.0:1789760777.543205:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.543218:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.543227:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.543235:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.543240:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.543251:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.543274:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.543282:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fae8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.543474:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515789952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25468us (26126us total) trans 4294973479 rc 0/0 00000100:00100000:3.0:1789760777.543485:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6191 00000100:00100000:3.0:1789760777.543494:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755950873372 time 755983277889 state 0x2 ref 0 00000100:00100000:0.0:1789760777.547050:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.547142:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790080 00000100:00100000:2.0:1789760777.547152:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790080 00000100:00100000:2.0:1789760777.547206:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755950873372 time 755983277889 state 0x0 ref 1 00000100:00100000:2.0:1789760777.547222:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755983538305 time 755983277889 state 0x1 ref 1 00000100:00100000:2.0:1789760777.547288:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755983538305 time 756012576633 state 0x0 ref 1 00000100:00100000:2.0:1789760777.547301:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.547353:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25243648, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.547437:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.547453:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.547458:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.547466:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.547480:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760777.570893:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25243648 -> 25247744 00010000:00010000:1.0:1789760777.570915:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.570933:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.570941:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.570949:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760777.570954:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.570966:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.570986:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.570994:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1faef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.571087:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515790080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23784us (24042us total) trans 4294973480 rc 0/0 00000100:00100000:1.0:1789760777.571098:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6192 00000100:00100000:1.0:1789760777.571107:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755983538305 time 756012576633 state 0x2 ref 0 00000001:00080000:2.0:1789760777.571188:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973480 is committed 00000100:00100000:1.0:1789760777.575522:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.575621:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790208 00000100:00100000:2.0:1789760777.575645:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790208 00000100:00100000:2.0:1789760777.575701:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 755983538305 time 756012576633 state 0x0 ref 1 00000100:00100000:2.0:1789760777.575770:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756012837049 time 756012576633 state 0x1 ref 1 00000100:00100000:2.0:1789760777.575793:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756012837049 time 756041082505 state 0x0 ref 1 00000100:00100000:2.0:1789760777.575806:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515790208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.575857:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25247744, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.575866:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.575883:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.575888:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.575894:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.575907:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.602787:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973481 is committed 00002000:00010000:2.0:1789760777.602915:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25247744 -> 25251840 00010000:00010000:2.0:1789760777.602930:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.602943:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.602952:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.602960:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.602965:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.602974:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.602993:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.603001:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1faf6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.603097:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515790208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27288us (27580us total) trans 4294973481 rc 0/0 00000100:00100000:2.0:1789760777.603110:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6193 00000100:00100000:2.0:1789760777.603121:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756012837049 time 756041082505 state 0x2 ref 0 00000100:00100000:3.0:1789760777.606155:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.606243:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790336 00000100:00100000:2.0:1789760777.606254:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790336 00000100:00100000:2.0:1789760777.606312:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756012837049 time 756041082505 state 0x0 ref 1 00000100:00100000:2.0:1789760777.606328:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756041342921 time 756041082505 state 0x1 ref 1 00000100:00100000:2.0:1789760777.606455:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756041342921 time 756071742229 state 0x0 ref 1 00000100:00100000:2.0:1789760777.606468:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.606518:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25251840, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.606528:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.606544:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.606549:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.606555:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.606568:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.673069:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973482 is committed 00002000:00010000:2.0:1789760777.673165:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25251840 -> 25255936 00010000:00010000:2.0:1789760777.673180:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.673194:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.673204:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.673211:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.673216:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.673225:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.673246:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.673254:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fafd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.673350:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66880us (67198us total) trans 4294973482 rc 0/0 00000100:00100000:2.0:1789760777.673511:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6194 00000100:00100000:2.0:1789760777.673522:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756041342921 time 756071742229 state 0x2 ref 0 00000100:00100000:3.0:1789760777.676844:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760777.677115:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790464 00000100:00100000:1.0:1789760777.677126:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790464 00000100:00100000:1.0:1789760777.677181:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756041342921 time 756071742229 state 0x0 ref 1 00000100:00100000:1.0:1789760777.677197:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756072002645 time 756071742229 state 0x1 ref 1 00000100:00100000:1.0:1789760777.677229:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756072002645 time 756142518373 state 0x0 ref 1 00000100:00100000:1.0:1789760777.677243:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515790464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760777.677291:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25255936, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:1.0:1789760777.677301:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.677316:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760777.677320:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.677327:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.677341:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.700254:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973483 is committed 00002000:00010000:3.0:1789760777.700418:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25255936 -> 25260032 00010000:00010000:3.0:1789760777.700435:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.700447:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.700456:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.700464:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.700469:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.700478:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.700500:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.700508:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fb04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.700608:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515790464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23361us (23767us total) trans 4294973483 rc 0/0 00000100:00100000:3.0:1789760777.700619:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6195 00000100:00100000:3.0:1789760777.700629:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756072002645 time 756142518373 state 0x2 ref 0 00000100:00100000:2.0:1789760777.704161:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760777.704262:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790592 00000100:00100000:0.0:1789760777.704273:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790592 00000100:00100000:0.0:1789760777.704323:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756072002645 time 756142518373 state 0x0 ref 1 00000100:00100000:0.0:1789760777.704338:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756142778789 time 756142518373 state 0x1 ref 1 00000100:00100000:0.0:1789760777.704373:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756142778789 time 756169661867 state 0x0 ref 1 00000100:00100000:0.0:1789760777.704385:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760777.704431:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25260032, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:0.0:1789760777.704445:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.704461:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760777.704466:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.704473:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.704487:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.732858:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544419200:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.732925:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.733110:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544419072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.733143:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544419712:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.733145:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.733173:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.733174:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544419584:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.733196:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.733198:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760777.733203:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544419328:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.733212:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760777.733218:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544419456:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.733224:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.733236:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.733268:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419200 00000100:00100000:1.0:1789760777.733275:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419200 00000100:00100000:1.0:1789760777.733313:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419584 00000100:00100000:1.0:1789760777.733316:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419584 00000100:00100000:2.0:1789760777.733320:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419072 00000100:00100000:2.0:1789760777.733323:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419072 00000100:00100000:1.0:1789760777.733339:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419328 00000100:00100000:1.0:1789760777.733342:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419328 00000100:00100000:2.0:1789760777.733348:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760777.733355:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:1.0:1789760777.733361:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544419200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.733472:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:3.0:1789760777.733481:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544419584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.733526:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544419200:12345-0@lo:400:kworker.0 Request processed in 165us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.733526:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544419584:12345-0@lo:400:kworker.0 Request processed in 45us (328us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.733533:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:2.0:1789760777.733552:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544419072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.733583:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544419584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760777.733584:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544419072:12345-0@lo:400:kworker.0 Request processed in 230us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760777.733590:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1789760777.733601:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544419072:0@lo:400:kworker.0 00000100:00080000:0.0:1789760777.733851:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760777.733981:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00100000:3.0:1789760777.733987:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:3.0:1789760777.733996:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544419328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.734014:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544419200:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.734114:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c63800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544419328:12345-0@lo:400:kworker.0 Request processed in 120us (888us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.734122:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:3.0:1789760777.734142:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419712 00000100:00100000:3.0:1789760777.734146:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419712 00000100:00100000:3.0:1789760777.734181:0:6706:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419456 00000100:00100000:3.0:1789760777.734184:0:6706:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419456 00000100:00080000:2.0:1789760777.734195:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760777.734198:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760777.734205:0:6706:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1789760777.734212:0:6706:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544419712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.734249:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544419328:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.734276:0:6706:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544419712:12345-0@lo:400:kworker.0 Request processed in 64us (1102us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.734284:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:3.0:1789760777.734288:0:6706:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:0.0:1789760777.734291:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544419456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.734333:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544419712:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.734440:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544419456:12345-0@lo:400:kworker.0 Request processed in 149us (1202us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.734447:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1789760777.734529:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544419456:0@lo:400:kworker.0 00002000:00010000:3.0:1789760777.735194:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25260032 -> 25264128 00010000:00010000:3.0:1789760777.735208:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.735220:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.735234:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.735242:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.735247:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.735256:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.735275:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.735285:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fb0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.735361:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515790592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30975us (31210us total) trans 4294973484 rc 0/0 00000100:00100000:3.0:1789760777.735370:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6196 00000100:00100000:3.0:1789760777.735381:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756142778789 time 756169661867 state 0x2 ref 0 00000001:00080000:1.0:1789760777.735872:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973484 is committed 00000100:00100000:0.0:1789760777.739523:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760777.739608:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790720 00000100:00100000:3.0:1789760777.739620:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790720 00000100:00100000:3.0:1789760777.739674:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756142778789 time 756169661867 state 0x0 ref 1 00000100:00100000:3.0:1789760777.739690:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756169922283 time 756169661867 state 0x1 ref 1 00000100:00100000:2.0:1789760777.739764:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756169922283 time 756205050224 state 0x0 ref 1 00000100:00100000:2.0:1789760777.739840:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.739889:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25264128, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.739898:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.739915:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.739920:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.739926:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.739940:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.774364:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25264128 -> 25268224 00010000:00010000:3.0:1789760777.774385:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.774399:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.774409:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.774416:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.774421:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.774431:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.774451:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.774459:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.774549:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515790720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34710us (35031us total) trans 4294973485 rc 0/0 00000100:00100000:3.0:1789760777.774559:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6197 00000100:00100000:3.0:1789760777.774569:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756169922283 time 756205050224 state 0x2 ref 0 00000001:00080000:1.0:1789760777.776430:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973485 is committed 00000100:00100000:0.0:1789760777.783067:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.783179:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790848 00000100:00100000:2.0:1789760777.783190:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790848 00000100:00100000:2.0:1789760777.783239:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756169922283 time 756205050224 state 0x0 ref 1 00000100:00100000:2.0:1789760777.783253:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756205310640 time 756205050224 state 0x1 ref 1 00000100:00100000:2.0:1789760777.783317:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756205310640 time 756248606039 state 0x0 ref 1 00000100:00100000:2.0:1789760777.783330:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515790848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.783377:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25268224, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.783386:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.783402:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.783407:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.783412:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.783427:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.814309:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973486 is committed 00002000:00010000:2.0:1789760777.814372:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25268224 -> 25272320 00010000:00010000:2.0:1789760777.814387:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.814400:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.814409:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.814418:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.814423:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.814432:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.814451:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.814459:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.814543:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515790848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31211us (31480us total) trans 4294973486 rc 0/0 00000100:00100000:2.0:1789760777.814553:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6198 00000100:00100000:2.0:1789760777.814563:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756205310640 time 756248606039 state 0x2 ref 0 00000100:00100000:0.0:1789760777.820425:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.820462:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515790976 00000100:00100000:2.0:1789760777.820472:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515790976 00000100:00100000:2.0:1789760777.820533:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756205310640 time 756248606039 state 0x0 ref 1 00000100:00100000:2.0:1789760777.820551:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756248866455 time 756248606039 state 0x1 ref 1 00000100:00100000:2.0:1789760777.820578:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756248866455 time 756285867581 state 0x0 ref 1 00000100:00100000:2.0:1789760777.820591:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.820638:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25272320, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc807 00010000:00010000:2.0:1789760777.820646:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.820661:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.820665:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.820672:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.820687:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.847437:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973487 is committed 00002000:00010000:2.0:1789760777.847540:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25272320 -> 25276416 00002000:00010000:2.0:1789760777.847545:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:2.0:1789760777.847548:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00010000:00010000:2.0:1789760777.847561:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.847574:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.847582:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.847593:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.847597:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.847605:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.847625:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.847697:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.847812:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515790976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27219us (27393us total) trans 4294973487 rc 0/0 00000100:00100000:2.0:1789760777.847822:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6199 00000100:00100000:2.0:1789760777.847831:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756248866455 time 756285867581 state 0x2 ref 0 00000100:00100000:0.0:1789760777.851802:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760777.851883:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791104 00000100:00100000:3.0:1789760777.851892:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791104 00000100:00100000:3.0:1789760777.851942:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756248866455 time 756285867581 state 0x0 ref 1 00000100:00100000:3.0:1789760777.851956:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756286127997 time 756285867581 state 0x1 ref 1 00000100:00100000:3.0:1789760777.851983:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756286127997 time 756317272063 state 0x0 ref 1 00000100:00100000:3.0:1789760777.851995:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515791104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760777.852040:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25276416, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760777.852049:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.852076:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.852081:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.852086:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.852100:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.885010:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973488 is committed 00002000:00010000:2.0:1789760777.885246:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25276416 -> 25280512 00010000:00010000:2.0:1789760777.885262:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.885275:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.885288:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.885294:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.885298:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.885306:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.885324:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.885331:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fb27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.885543:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515791104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33543us (33744us total) trans 4294973488 rc 0/0 00000100:00100000:2.0:1789760777.885555:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6200 00000100:00100000:2.0:1789760777.885564:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756286127997 time 756317272063 state 0x2 ref 0 00000100:00100000:0.0:1789760777.891164:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760777.891320:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791232 00000100:00100000:3.0:1789760777.891331:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791232 00000100:00100000:3.0:1789760777.891451:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756286127997 time 756317272063 state 0x0 ref 1 00000100:00100000:3.0:1789760777.891470:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756317532479 time 756317272063 state 0x1 ref 1 00000100:00100000:3.0:1789760777.891503:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756317532479 time 756356792597 state 0x0 ref 1 00000100:00100000:3.0:1789760777.891516:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515791232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760777.891562:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25280512, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760777.891571:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.891586:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.891590:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.891596:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.891610:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.934027:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973489 is committed 00002000:00010000:3.0:1789760777.934070:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25280512 -> 25284608 00010000:00010000:3.0:1789760777.934097:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.934110:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.934119:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.934132:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760777.934137:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.934146:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.934167:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.934175:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.934438:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515791232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42919us (43277us total) trans 4294973489 rc 0/0 00000100:00100000:3.0:1789760777.934450:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6201 00000100:00100000:3.0:1789760777.934460:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756317532479 time 756356792597 state 0x2 ref 0 00000100:00100000:0.0:1789760777.938311:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760777.938549:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791360 00000100:00100000:3.0:1789760777.938560:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791360 00000100:00100000:3.0:1789760777.938614:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756317532479 time 756356792597 state 0x0 ref 1 00000100:00100000:3.0:1789760777.938695:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756357053013 time 756356792597 state 0x1 ref 1 00000100:00100000:3.0:1789760777.938947:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756357053013 time 756404234785 state 0x0 ref 1 00000100:00100000:3.0:1789760777.938965:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515791360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760777.939015:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25284608, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760777.939023:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.939038:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.939042:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.939048:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.939060:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.979962:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25284608 -> 25288704 00010000:00010000:2.0:1789760777.979984:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.980004:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.980014:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.980021:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760777.980026:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.980035:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.980055:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.980062:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fb35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.980165:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515791360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41198us (41859us total) trans 4294973490 rc 0/0 00000100:00100000:2.0:1789760777.980176:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6202 00000100:00100000:2.0:1789760777.980185:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756357053013 time 756404234785 state 0x2 ref 0 00000001:00080000:1.0:1789760777.981858:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973490 is committed 00000100:00100000:0.0:1789760777.985319:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760777.985538:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791488 00000100:00100000:2.0:1789760777.985549:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791488 00000100:00100000:2.0:1789760777.985602:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756357053013 time 756404234785 state 0x0 ref 1 00000100:00100000:2.0:1789760777.985617:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756404495201 time 756404234785 state 0x1 ref 1 00000100:00100000:2.0:1789760777.985853:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756404495201 time 756451140729 state 0x0 ref 1 00000100:00100000:2.0:1789760777.985868:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515791488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760777.985920:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25288704, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760777.985929:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.985947:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.985952:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.985957:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.985971:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.020735:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25288704 -> 25292800 00010000:00010000:1.0:1789760778.020757:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.020771:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.020786:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.020793:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.020798:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.020808:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.020828:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.020835:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fb3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.020927:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515791488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35058us (35674us total) trans 4294973491 rc 0/0 00000100:00100000:1.0:1789760778.020937:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6203 00000100:00100000:1.0:1789760778.020946:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756404495201 time 756451140729 state 0x2 ref 0 00000001:00080000:1.0:1789760778.020982:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973491 is committed 00000100:00100000:2.0:1789760778.024688:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760778.025076:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791616 00000100:00100000:0.0:1789760778.025086:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791616 00000100:00100000:0.0:1789760778.025193:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756404495201 time 756451140729 state 0x0 ref 1 00000100:00100000:0.0:1789760778.025210:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756451401145 time 756451140729 state 0x1 ref 1 00000100:00100000:0.0:1789760778.025238:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756451401145 time 756490527955 state 0x0 ref 1 00000100:00100000:0.0:1789760778.025251:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515791616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760778.025300:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25292800, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:0.0:1789760778.025310:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.025326:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760778.025331:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.025337:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.025360:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760778.026089:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760778.068788:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25292800 -> 25296896 00010000:00010000:0.0:1789760778.068809:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.068823:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.068837:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.068844:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760778.068849:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.068858:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.068876:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.068885:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fb43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.068971:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515791616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43717us (44289us total) trans 4294973492 rc 0/0 00000100:00100000:0.0:1789760778.068982:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6204 00000100:00100000:0.0:1789760778.068991:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756451401145 time 756490527955 state 0x2 ref 0 00000001:00080000:2.0:1789760778.069387:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973492 is committed 00000100:00100000:0.0:1789760778.072395:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760778.072870:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791744 00000100:00100000:2.0:1789760778.072882:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791744 00000100:00100000:2.0:1789760778.072937:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756451401145 time 756490527955 state 0x0 ref 1 00000100:00100000:2.0:1789760778.072953:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756490788371 time 756490527955 state 0x1 ref 1 00000100:00100000:2.0:1789760778.072987:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756490788371 time 756538275351 state 0x0 ref 1 00000100:00100000:2.0:1789760778.072999:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515791744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.073045:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25296896, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.073054:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.073476:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760778.073481:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.073488:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.073502:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.117180:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973493 is committed 00002000:00010000:1.0:1789760778.117576:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25296896 -> 25300992 00010000:00010000:1.0:1789760778.117597:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117618:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117629:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117973:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.117980:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117991:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.118126:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.118138:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fb4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.118336:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515791744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45329us (45941us total) trans 4294973493 rc 0/0 00000100:00100000:1.0:1789760778.118349:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6205 00000100:00100000:1.0:1789760778.118385:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756490788371 time 756538275351 state 0x2 ref 0 00000100:00100000:0.0:1789760778.124454:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.124587:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515791872 00000100:00100000:3.0:1789760778.124597:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515791872 00000100:00100000:3.0:1789760778.125136:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756490788371 time 756538275351 state 0x0 ref 1 00000100:00100000:3.0:1789760778.125152:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756538535767 time 756538275351 state 0x1 ref 1 00000100:00100000:2.0:1789760778.125534:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756538535767 time 756590819637 state 0x0 ref 1 00000100:00100000:2.0:1789760778.125555:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515791872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.125613:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25300992, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.125623:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.125651:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760778.125656:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.125663:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.125679:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.175688:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760778.175764:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760778.177448:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760778.177488:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792000 00000100:00100000:0.0:1789760778.177500:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792000 00000100:00100000:0.0:1789760778.177551:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 406 00000100:00100000:0.0:1789760778.177561:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515792000:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760778.177645:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515792000:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 84us (1880us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760778.177653:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 406 00000100:00100000:1.0:1789760778.178023:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792128 00000100:00100000:1.0:1789760778.178032:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792128 00000100:00100000:1.0:1789760778.178091:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792256 00000100:00100000:1.0:1789760778.178097:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792256 00000100:00100000:1.0:1789760778.178125:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 491 00000100:00100000:1.0:1789760778.178133:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515792128:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.178172:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515792128:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 39us (2555us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.178179:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 491 00000100:00100000:1.0:1789760778.178200:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 492 00000100:00100000:1.0:1789760778.178210:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515792256:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.178248:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806248a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515792256:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 39us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.178255:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 492 00000100:00080000:1.0:1789760778.178278:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760778.185923:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973494 is committed 00002000:00010000:3.0:1789760778.186047:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25300992 -> 25305088 00010000:00010000:3.0:1789760778.186065:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.186079:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.186094:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.186101:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760778.186106:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.186115:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.186137:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.186145:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fb51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.186237:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515791872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 60686us (61788us total) trans 4294973494 rc 0/0 00000100:00100000:3.0:1789760778.186247:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6206 00000100:00100000:3.0:1789760778.186256:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756538535767 time 756590819637 state 0x2 ref 0 00000100:00100000:0.0:1789760778.193235:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.193371:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792384 00000100:00100000:3.0:1789760778.193382:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792384 00000100:00100000:3.0:1789760778.193442:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756538535767 time 756590819637 state 0x0 ref 1 00000100:00100000:3.0:1789760778.193459:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756591080053 time 756590819637 state 0x1 ref 1 00000100:00100000:3.0:1789760778.193501:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756591080053 time 756658788383 state 0x0 ref 1 00000100:00100000:3.0:1789760778.193518:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515792384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760778.193581:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25305088, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760778.193593:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.193610:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760778.193615:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.193622:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.193654:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.225971:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973495 is committed 00002000:00010000:3.0:1789760778.226135:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25305088 -> 25309184 00010000:00010000:3.0:1789760778.226149:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.226162:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.226171:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.226178:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760778.226184:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.226193:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.226212:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.226220:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.226346:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515792384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32827us (33116us total) trans 4294973495 rc 0/0 00000100:00100000:3.0:1789760778.226424:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6207 00000100:00100000:3.0:1789760778.226435:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756591080053 time 756658788383 state 0x2 ref 0 00000100:00100000:0.0:1789760778.238507:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.241136:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792512 00000100:00100000:3.0:1789760778.241150:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792512 00000100:00100000:2.0:1789760778.241568:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756591080053 time 756658788383 state 0x0 ref 1 00000100:00100000:2.0:1789760778.241584:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756659048799 time 756658788383 state 0x1 ref 1 00000100:00100000:2.0:1789760778.241618:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756659048799 time 756706907231 state 0x0 ref 1 00000100:00100000:2.0:1789760778.241778:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515792512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.241830:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25309184, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.241840:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.241857:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760778.241862:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.241869:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.241882:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.276311:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973496 is committed 00002000:00010000:1.0:1789760778.276500:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25309184 -> 25313280 00010000:00010000:1.0:1789760778.276516:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.276529:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.276538:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.276546:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.276551:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.276560:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.276580:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.276588:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fb5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.276829:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515792512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35194us (38326us total) trans 4294973496 rc 0/0 00000100:00100000:1.0:1789760778.276841:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6208 00000100:00100000:1.0:1789760778.276851:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756659048799 time 756706907231 state 0x2 ref 0 00000100:00100000:2.0:1789760778.280187:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.280287:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792640 00000100:00100000:1.0:1789760778.280299:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792640 00000100:00100000:1.0:1789760778.280431:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756659048799 time 756706907231 state 0x0 ref 1 00000100:00100000:1.0:1789760778.280448:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756707167647 time 756706907231 state 0x1 ref 1 00000100:00100000:1.0:1789760778.280511:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756707167647 time 756745799806 state 0x0 ref 1 00000100:00100000:1.0:1789760778.280525:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515792640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760778.280574:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25313280, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:1.0:1789760778.280584:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.280599:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760778.280603:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.280610:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.280623:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.314464:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973497 is committed 00002000:00010000:1.0:1789760778.314540:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25313280 -> 25317376 00010000:00010000:1.0:1789760778.314558:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314572:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314581:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314587:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.314593:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314603:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314624:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.314692:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.314815:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515792640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34287us (34633us total) trans 4294973497 rc 0/0 00000100:00100000:1.0:1789760778.314825:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6209 00000100:00100000:1.0:1789760778.314837:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756707167647 time 756745799806 state 0x2 ref 0 00000100:00100000:0.0:1789760778.319547:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760778.319653:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792768 00000100:00100000:2.0:1789760778.319665:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792768 00000100:00100000:2.0:1789760778.319762:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756707167647 time 756745799806 state 0x0 ref 1 00000100:00100000:2.0:1789760778.319777:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756746060222 time 756745799806 state 0x1 ref 1 00000100:00100000:2.0:1789760778.319806:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756746060222 time 756785095384 state 0x0 ref 1 00000100:00100000:2.0:1789760778.319818:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515792768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.319868:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25317376, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.319877:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.319892:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760778.319896:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.319903:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.319916:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.374583:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25317376 -> 25321472 00010000:00010000:1.0:1789760778.374603:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.374617:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.374626:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.374645:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.374650:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.374664:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.374683:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.374692:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fb6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.374818:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515792768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54996us (55274us total) trans 4294973498 rc 0/0 00000100:00100000:1.0:1789760778.374830:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6210 00000100:00100000:1.0:1789760778.374840:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756746060222 time 756785095384 state 0x2 ref 0 00000001:00080000:1.0:1789760778.374882:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973498 is committed 00000100:00100000:1.0:1789760778.382256:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760778.382383:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515792896 00000100:00100000:2.0:1789760778.382394:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515792896 00000100:00100000:2.0:1789760778.382448:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756746060222 time 756785095384 state 0x0 ref 1 00000100:00100000:2.0:1789760778.382465:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756785355800 time 756785095384 state 0x1 ref 1 00000100:00100000:2.0:1789760778.382493:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756785355800 time 756847782776 state 0x0 ref 1 00000100:00100000:2.0:1789760778.382509:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515792896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.382556:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25321472, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.382566:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.382582:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760778.382586:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.382592:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.382606:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.416276:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973499 is committed 00002000:00010000:1.0:1789760778.416373:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25321472 -> 25325568 00010000:00010000:1.0:1789760778.416388:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.416401:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.416409:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.416421:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.416426:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.416434:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.416453:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.416461:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fb74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.416550:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515792896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34038us (34298us total) trans 4294973499 rc 0/0 00000100:00100000:1.0:1789760778.416561:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6211 00000100:00100000:1.0:1789760778.416570:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756785355800 time 756847782776 state 0x2 ref 0 00000100:00100000:2.0:1789760778.427187:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.427306:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793024 00000100:00100000:3.0:1789760778.427317:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793024 00000100:00100000:3.0:1789760778.427370:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756785355800 time 756847782776 state 0x0 ref 1 00000100:00100000:3.0:1789760778.427385:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756848043192 time 756847782776 state 0x1 ref 1 00000100:00100000:3.0:1789760778.427420:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756848043192 time 756892708730 state 0x0 ref 1 00000100:00100000:3.0:1789760778.427441:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760778.427487:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25325568, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760778.427495:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.427515:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760778.427520:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.427526:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.427539:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.458176:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973500 is committed 00002000:00010000:1.0:1789760778.458282:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25325568 -> 25329664 00010000:00010000:1.0:1789760778.458298:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.458311:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.458327:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.458335:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.458340:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.458359:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.458377:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.458384:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fb7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.458476:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31031us (31292us total) trans 4294973500 rc 0/0 00000100:00100000:1.0:1789760778.458486:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6212 00000100:00100000:1.0:1789760778.458497:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756848043192 time 756892708730 state 0x2 ref 0 00000100:00100000:0.0:1789760778.464704:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.464805:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793152 00000100:00100000:3.0:1789760778.464817:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793152 00000100:00100000:3.0:1789760778.464871:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756848043192 time 756892708730 state 0x0 ref 1 00000100:00100000:3.0:1789760778.464886:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756892969146 time 756892708730 state 0x1 ref 1 00000100:00100000:3.0:1789760778.464907:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756892969146 time 756930196476 state 0x0 ref 1 00000100:00100000:3.0:1789760778.464918:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760778.464967:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25329664, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760778.464977:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.464998:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760778.465002:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.465020:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.465035:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.536597:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973501 is committed 00002000:00010000:1.0:1789760778.536857:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25329664 -> 25333760 00010000:00010000:1.0:1789760778.536873:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.536891:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.536963:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.536970:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.536975:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.536984:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.537004:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.537011:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fb82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.537193:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 72272us (72495us total) trans 4294973501 rc 0/0 00000100:00100000:1.0:1789760778.537204:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6213 00000100:00100000:1.0:1789760778.537214:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756892969146 time 756930196476 state 0x2 ref 0 00000100:00100000:0.0:1789760778.541599:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.542604:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793280 00000100:00100000:1.0:1789760778.542615:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793280 00000100:00100000:2.0:1789760778.542866:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756892969146 time 756930196476 state 0x0 ref 1 00000100:00100000:2.0:1789760778.542882:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756930456892 time 756930196476 state 0x1 ref 1 00000100:00100000:2.0:1789760778.542942:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756930456892 time 757008229881 state 0x0 ref 1 00000100:00100000:2.0:1789760778.542956:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515793280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.543004:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25333760, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.543012:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.543032:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.543037:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.543043:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.543057:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.560067:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760778.596151:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25333760 -> 25337856 00010000:00010000:2.0:1789760778.596170:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.596183:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.596341:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.596349:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760778.596354:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.596364:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.596388:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.596396:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fb89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.596446:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973502 is committed 00000100:00100000:2.0:1789760778.596481:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515793280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53524us (54888us total) trans 4294973502 rc 0/0 00000100:00100000:2.0:1789760778.596490:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6214 00000100:00100000:2.0:1789760778.596499:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756930456892 time 757008229881 state 0x2 ref 0 00000100:00100000:2.0:1789760778.604138:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.604607:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793408 00000100:00100000:1.0:1789760778.604617:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793408 00000100:00100000:1.0:1789760778.604684:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 756930456892 time 757008229881 state 0x0 ref 1 00000100:00100000:1.0:1789760778.604699:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757008490297 time 757008229881 state 0x1 ref 1 00000100:00100000:1.0:1789760778.604759:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757008490297 time 757070047775 state 0x0 ref 1 00000100:00100000:1.0:1789760778.604777:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515793408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760778.604827:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25337856, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:1.0:1789760778.604835:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.604850:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.604853:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.604859:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.604872:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.642851:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973503 is committed 00002000:00010000:2.0:1789760778.643492:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25337856 -> 25341952 00010000:00010000:2.0:1789760778.643508:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.643521:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.643530:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.643538:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760778.643543:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.643557:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.643576:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.643582:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fb90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.643971:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515793408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39195us (39836us total) trans 4294973503 rc 0/0 00000100:00100000:2.0:1789760778.643982:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6215 00000100:00100000:2.0:1789760778.643992:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757008490297 time 757070047775 state 0x2 ref 0 00000100:00100000:0.0:1789760778.646783:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.646868:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793536 00000100:00100000:1.0:1789760778.646878:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793536 00000100:00100000:1.0:1789760778.646932:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757008490297 time 757070047775 state 0x0 ref 1 00000100:00100000:1.0:1789760778.646946:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757070308191 time 757070047775 state 0x1 ref 1 00000100:00100000:1.0:1789760778.646982:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757070308191 time 757112271064 state 0x0 ref 1 00000100:00100000:1.0:1789760778.646994:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515793536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760778.647040:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25341952, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:1.0:1789760778.647054:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.647070:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.647075:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.647081:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.647096:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760778.671885:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25341952 -> 25346048 00010000:00010000:2.0:1789760778.671908:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.671921:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.671931:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.671937:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760778.671943:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.671952:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.671971:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.671980:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fb97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.672076:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515793536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25080us (25375us total) trans 4294973504 rc 0/0 00000100:00100000:2.0:1789760778.672087:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6216 00000100:00100000:2.0:1789760778.672097:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757070308191 time 757112271064 state 0x2 ref 0 00000001:00080000:0.0:1789760778.672253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973504 is committed 00000100:00100000:2.0:1789760778.675979:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.676271:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793664 00000100:00100000:3.0:1789760778.676282:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793664 00000100:00100000:3.0:1789760778.676334:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757070308191 time 757112271064 state 0x0 ref 1 00000100:00100000:3.0:1789760778.676358:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757112531480 time 757112271064 state 0x1 ref 1 00000100:00100000:3.0:1789760778.676386:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757112531480 time 757141675227 state 0x0 ref 1 00000100:00100000:3.0:1789760778.676398:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760778.676444:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25346048, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760778.676453:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.676468:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.676473:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.676479:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.676497:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.704260:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973505 is committed 00002000:00010000:3.0:1789760778.704423:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25346048 -> 25350144 00010000:00010000:3.0:1789760778.704439:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704451:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704460:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704467:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760778.704473:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704482:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704501:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704509:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fb9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.704605:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28203us (28630us total) trans 4294973505 rc 0/0 00000100:00100000:3.0:1789760778.704616:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6217 00000100:00100000:3.0:1789760778.704691:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757112531480 time 757141675227 state 0x2 ref 0 00000100:00100000:2.0:1789760778.708153:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.708226:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793792 00000100:00100000:3.0:1789760778.708236:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793792 00000100:00100000:3.0:1789760778.708282:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757112531480 time 757141675227 state 0x0 ref 1 00000100:00100000:3.0:1789760778.708296:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757141935643 time 757141675227 state 0x1 ref 1 00000100:00100000:3.0:1789760778.708326:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757141935643 time 757173615458 state 0x0 ref 1 00000100:00100000:3.0:1789760778.708338:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515793792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760778.708502:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25350144, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760778.708512:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.708527:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.708532:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.708539:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.708553:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.735032:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973506 is committed 00002000:00010000:3.0:1789760778.735130:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25350144 -> 25354240 00010000:00010000:3.0:1789760778.735145:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.735157:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.735166:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.735173:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760778.735178:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.735187:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.735207:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.735215:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fba5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.735341:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515793792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27000us (27194us total) trans 4294973506 rc 0/0 00000100:00100000:3.0:1789760778.735365:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6218 00000100:00100000:3.0:1789760778.735374:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757141935643 time 757173615458 state 0x2 ref 0 00000100:00100000:0.0:1789760778.738235:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760778.738341:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515793920 00000100:00100000:3.0:1789760778.738404:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515793920 00000100:00100000:3.0:1789760778.738456:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757141935643 time 757173615458 state 0x0 ref 1 00000100:00100000:3.0:1789760778.738470:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757173875874 time 757173615458 state 0x1 ref 1 00000100:00100000:3.0:1789760778.738502:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757173875874 time 757203791152 state 0x0 ref 1 00000100:00100000:3.0:1789760778.738514:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760778.738559:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25354240, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:3.0:1789760778.738567:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.738581:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.738585:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.738592:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.738606:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.770311:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973507 is committed 00002000:00010000:1.0:1789760778.770608:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25354240 -> 25358336 00010000:00010000:1.0:1789760778.770683:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.770696:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.770796:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.770810:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.770814:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.770823:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.770841:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.770848:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fbac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.770930:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515793920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32414us (32699us total) trans 4294973507 rc 0/0 00000100:00100000:1.0:1789760778.770940:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6219 00000100:00100000:1.0:1789760778.770949:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757173875874 time 757203791152 state 0x2 ref 0 00000100:00100000:1.0:1789760778.777802:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760778.778497:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794048 00000100:00100000:2.0:1789760778.778509:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794048 00000100:00100000:2.0:1789760778.778561:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757173875874 time 757203791152 state 0x0 ref 1 00000100:00100000:2.0:1789760778.778577:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757204051568 time 757203791152 state 0x1 ref 1 00000100:00100000:2.0:1789760778.778604:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757204051568 time 757243893667 state 0x0 ref 1 00000100:00100000:2.0:1789760778.778618:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515794048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.778666:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25358336, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:2.0:1789760778.778674:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.778689:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.778693:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.778699:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.778758:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.813332:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973508 is committed 00002000:00010000:1.0:1789760778.813440:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25358336 -> 25362432 00010000:00010000:1.0:1789760778.813457:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.813469:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.813483:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.813491:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.813495:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.813505:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.813524:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.813532:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fbb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.813629:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515794048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35002us (35826us total) trans 4294973508 rc 0/0 00000100:00100000:1.0:1789760778.813639:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6220 00000100:00100000:1.0:1789760778.813649:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757204051568 time 757243893667 state 0x2 ref 0 00000100:00100000:2.0:1789760778.818219:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.818313:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794176 00000100:00100000:1.0:1789760778.818324:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794176 00000100:00100000:1.0:1789760778.818377:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757204051568 time 757243893667 state 0x0 ref 1 00000100:00100000:1.0:1789760778.818393:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757244154083 time 757243893667 state 0x1 ref 1 00000100:00100000:1.0:1789760778.818427:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757244154083 time 757283716038 state 0x0 ref 1 00000100:00100000:1.0:1789760778.818440:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515794176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760778.818489:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25362432, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc807 00010000:00010000:1.0:1789760778.818498:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.818514:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.818518:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.818524:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.818537:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.849261:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25362432 -> 25366528 00002000:00010000:3.0:1789760778.849271:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:3.0:1789760778.849332:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00010000:00010000:3.0:1789760778.849347:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.849360:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.849370:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.849377:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760778.849382:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.849391:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.849409:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.849417:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.849510:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515794176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31067us (31294us total) trans 4294973509 rc 0/0 00000100:00100000:3.0:1789760778.849521:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6221 00000100:00100000:3.0:1789760778.849532:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757244154083 time 757283716038 state 0x2 ref 0 00000001:00080000:0.0:1789760778.850915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973509 is committed 00000100:00100000:0.0:1789760778.855686:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760778.856215:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794304 00000100:00100000:2.0:1789760778.856225:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794304 00000100:00100000:2.0:1789760778.856281:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757244154083 time 757283716038 state 0x0 ref 1 00000100:00100000:2.0:1789760778.856296:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757283976454 time 757283716038 state 0x1 ref 1 00000100:00100000:2.0:1789760778.856364:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757283976454 time 757321653284 state 0x0 ref 1 00000100:00100000:2.0:1789760778.856376:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515794304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.856422:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25366528, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760778.856431:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.856446:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.856450:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.856456:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.856469:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.883285:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973510 is committed 00002000:00010000:2.0:1789760778.883372:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25366528 -> 25370624 00010000:00010000:2.0:1789760778.883389:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.883401:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.883409:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.883417:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760778.883421:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.883430:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.883450:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.883458:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fbc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.883582:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515794304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27202us (27900us total) trans 4294973510 rc 0/0 00000100:00100000:2.0:1789760778.883592:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6222 00000100:00100000:2.0:1789760778.883848:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757283976454 time 757321653284 state 0x2 ref 0 00000100:00100000:1.0:1789760778.887677:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760778.887819:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794432 00000100:00100000:2.0:1789760778.887828:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794432 00000100:00100000:2.0:1789760778.887887:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757283976454 time 757321653284 state 0x0 ref 1 00000100:00100000:2.0:1789760778.887901:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757321913700 time 757321653284 state 0x1 ref 1 00000100:00100000:2.0:1789760778.887931:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757321913700 time 757353219844 state 0x0 ref 1 00000100:00100000:2.0:1789760778.887943:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515794432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760778.887990:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25370624, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760778.887997:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.888012:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.888017:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.888022:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.888041:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.934148:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973511 is committed 00002000:00010000:3.0:1789760778.934415:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25370624 -> 25374720 00010000:00010000:3.0:1789760778.934430:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.934442:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.934451:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.934458:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760778.934464:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.934472:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.934491:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.934498:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fbc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.934788:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515794432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46679us (46952us total) trans 4294973511 rc 0/0 00000100:00100000:3.0:1789760778.934798:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6223 00000100:00100000:3.0:1789760778.934809:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757321913700 time 757353219844 state 0x2 ref 0 00000100:00100000:3.0:1789760778.943903:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.944077:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794560 00000100:00100000:1.0:1789760778.944087:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794560 00000100:00100000:1.0:1789760778.944136:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757321913700 time 757353219844 state 0x0 ref 1 00000100:00100000:1.0:1789760778.944150:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757353480260 time 757353219844 state 0x1 ref 1 00000100:00100000:1.0:1789760778.944177:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757353480260 time 757409466183 state 0x0 ref 1 00000100:00100000:1.0:1789760778.944191:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515794560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760778.944234:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25374720, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760778.944243:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.944258:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.944262:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.944267:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.944281:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.976554:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973512 is committed 00002000:00010000:1.0:1789760778.976746:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25374720 -> 25378816 00010000:00010000:1.0:1789760778.976762:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.976774:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.976783:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.976790:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760778.976795:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.976803:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.976822:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.976830:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.976911:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515794560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32716us (33012us total) trans 4294973512 rc 0/0 00000100:00100000:1.0:1789760778.976922:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6224 00000100:00100000:1.0:1789760778.976932:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757353480260 time 757409466183 state 0x2 ref 0 00000100:00100000:0.0:1789760778.980355:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760778.980477:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794688 00000100:00100000:1.0:1789760778.980488:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794688 00000100:00100000:1.0:1789760778.980536:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757353480260 time 757409466183 state 0x0 ref 1 00000100:00100000:1.0:1789760778.980550:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757409726599 time 757409466183 state 0x1 ref 1 00000100:00100000:1.0:1789760778.980582:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757409726599 time 757445871003 state 0x0 ref 1 00000100:00100000:1.0:1789760778.980594:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515794688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760778.980648:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25378816, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760778.980657:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.980675:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.980679:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.980685:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.980700:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.008728:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25378816 -> 25382912 00010000:00010000:2.0:1789760779.008749:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.008762:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.008773:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.008780:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.008786:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.008794:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.008813:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.008821:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1fbd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.008916:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515794688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28319us (28574us total) trans 4294973513 rc 0/0 00000100:00100000:2.0:1789760779.008927:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6225 00000100:00100000:2.0:1789760779.008937:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757409726599 time 757445871003 state 0x2 ref 0 00000001:00080000:0.0:1789760779.009017:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973513 is committed 00000100:00100000:2.0:1789760779.012506:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.012636:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794816 00000100:00100000:1.0:1789760779.012646:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794816 00000100:00100000:1.0:1789760779.012698:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757409726599 time 757445871003 state 0x0 ref 1 00000100:00100000:1.0:1789760779.012745:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757446131419 time 757445871003 state 0x1 ref 1 00000100:00100000:1.0:1789760779.012774:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757446131419 time 757478063757 state 0x0 ref 1 00000100:00100000:1.0:1789760779.012785:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515794816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.012835:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25382912, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.012843:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012857:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760779.012861:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.012868:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012886:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.042994:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25382912 -> 25387008 00010000:00010000:2.0:1789760779.043016:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.043030:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.043040:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.043049:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.043054:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.043063:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.043084:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.043097:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fbdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.043184:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515794816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30395us (30682us total) trans 4294973514 rc 0/0 00000100:00100000:2.0:1789760779.043195:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6226 00000100:00100000:2.0:1789760779.043205:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757446131419 time 757478063757 state 0x2 ref 0 00000001:00080000:3.0:1789760779.046528:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973514 is committed 00000100:00100000:2.0:1789760779.046690:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760779.046834:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515794944 00000100:00100000:3.0:1789760779.046844:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515794944 00000100:00100000:3.0:1789760779.046897:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757446131419 time 757478063757 state 0x0 ref 1 00000100:00100000:3.0:1789760779.046915:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757478324173 time 757478063757 state 0x1 ref 1 00000100:00100000:3.0:1789760779.046953:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757478324173 time 757512241798 state 0x0 ref 1 00000100:00100000:3.0:1789760779.046967:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515794944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760779.047023:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25387008, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:3.0:1789760779.047033:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.047050:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760779.047054:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.047061:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.047077:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.073101:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973515 is committed 00002000:00010000:1.0:1789760779.073288:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25387008 -> 25391104 00010000:00010000:1.0:1789760779.073304:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.073317:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.073326:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.073333:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.073339:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.073348:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.073422:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.073430:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fbe4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.073521:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515794944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26551us (26829us total) trans 4294973515 rc 0/0 00000100:00100000:1.0:1789760779.073532:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6227 00000100:00100000:1.0:1789760779.073542:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757478324173 time 757512241798 state 0x2 ref 0 00000100:00100000:0.0:1789760779.077577:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760779.077753:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795072 00000100:00100000:2.0:1789760779.077763:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795072 00000100:00100000:2.0:1789760779.077816:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757478324173 time 757512241798 state 0x0 ref 1 00000100:00100000:2.0:1789760779.077831:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757512502214 time 757512241798 state 0x1 ref 1 00000100:00100000:2.0:1789760779.077865:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757512502214 time 757543154352 state 0x0 ref 1 00000100:00100000:2.0:1789760779.077879:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515795072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.077932:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25391104, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760779.077942:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.077958:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760779.077963:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.077968:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.077983:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.109565:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973516 is committed 00002000:00010000:2.0:1789760779.109680:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25391104 -> 25395200 00010000:00010000:2.0:1789760779.109694:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.109735:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.109745:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.109753:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.109758:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.109767:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.109788:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.109796:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.109892:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515795072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32011us (32320us total) trans 4294973516 rc 0/0 00000100:00100000:2.0:1789760779.109903:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6228 00000100:00100000:2.0:1789760779.109912:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757512502214 time 757543154352 state 0x2 ref 0 00000100:00100000:2.0:1789760779.114911:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760779.115028:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795200 00000100:00100000:0.0:1789760779.115040:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795200 00000100:00100000:0.0:1789760779.115109:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757512502214 time 757543154352 state 0x0 ref 1 00000100:00100000:0.0:1789760779.115127:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757543414768 time 757543154352 state 0x1 ref 1 00000100:00100000:0.0:1789760779.115154:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757543414768 time 757580443357 state 0x0 ref 1 00000100:00100000:0.0:1789760779.115168:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515795200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760779.115219:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25395200, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:0.0:1789760779.115228:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.115245:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760779.115249:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.115256:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.115269:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.152106:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25395200 -> 25399296 00010000:00010000:1.0:1789760779.152131:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.152161:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.152170:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.152186:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.152191:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.152200:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.152224:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.152237:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fbf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.152333:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515795200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37162us (37427us total) trans 4294973517 rc 0/0 00000100:00100000:1.0:1789760779.152344:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6229 00000100:00100000:1.0:1789760779.152353:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757543414768 time 757580443357 state 0x2 ref 0 00000001:00080000:2.0:1789760779.153572:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973517 is committed 00000100:00100000:3.0:1789760779.155691:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.156020:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795328 00000100:00100000:1.0:1789760779.156030:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795328 00000100:00100000:2.0:1789760779.156213:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757543414768 time 757580443357 state 0x0 ref 1 00000100:00100000:2.0:1789760779.156231:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757580703773 time 757580443357 state 0x1 ref 1 00000100:00100000:2.0:1789760779.156292:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757580703773 time 757621580381 state 0x0 ref 1 00000100:00100000:2.0:1789760779.156306:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515795328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.156355:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25399296, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760779.156363:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.156376:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760779.156380:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.156384:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.156395:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.200737:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973518 is committed 00002000:00010000:1.0:1789760779.200902:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25399296 -> 25403392 00010000:00010000:1.0:1789760779.200917:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200931:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200940:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200948:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.200953:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200962:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200982:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.200990:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fbf9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.201087:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515795328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44778us (45399us total) trans 4294973518 rc 0/0 00000100:00100000:1.0:1789760779.201097:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6230 00000100:00100000:1.0:1789760779.201106:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757580703773 time 757621580381 state 0x2 ref 0 00000100:00100000:3.0:1789760779.208476:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.208889:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795456 00000100:00100000:1.0:1789760779.208900:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795456 00000100:00100000:1.0:1789760779.208954:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757580703773 time 757621580381 state 0x0 ref 1 00000100:00100000:1.0:1789760779.208969:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757621840797 time 757621580381 state 0x1 ref 1 00000100:00100000:1.0:1789760779.209007:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757621840797 time 757674295784 state 0x0 ref 1 00000100:00100000:1.0:1789760779.209020:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515795456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.209066:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25403392, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.209075:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.209091:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760779.209096:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.209102:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.209115:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.247776:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25403392 -> 25407488 00010000:00010000:1.0:1789760779.247798:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.247810:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.247818:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.247826:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.247831:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.247841:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.247861:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.247869:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.247967:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515795456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38944us (39495us total) trans 4294973519 rc 0/0 00000100:00100000:1.0:1789760779.247979:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6231 00000100:00100000:1.0:1789760779.247988:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757621840797 time 757674295784 state 0x2 ref 0 00000001:00080000:2.0:1789760779.248597:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973519 is committed 00000100:00100000:2.0:1789760779.259314:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.259458:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795584 00000100:00100000:1.0:1789760779.259468:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795584 00000100:00100000:1.0:1789760779.259520:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757621840797 time 757674295784 state 0x0 ref 1 00000100:00100000:1.0:1789760779.259535:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757674556200 time 757674295784 state 0x1 ref 1 00000100:00100000:1.0:1789760779.259567:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757674556200 time 757724855640 state 0x0 ref 1 00000100:00100000:1.0:1789760779.259578:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515795584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.259637:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25407488, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.259646:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.259662:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760779.259667:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.259672:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.259686:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.308069:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25407488 -> 25411584 00010000:00010000:3.0:1789760779.308088:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308100:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308108:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308114:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760779.308118:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308126:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308145:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308151:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fc07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.308245:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515795584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48663us (48935us total) trans 4294973520 rc 0/0 00000100:00100000:3.0:1789760779.308258:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6232 00000100:00100000:3.0:1789760779.308267:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757674556200 time 757724855640 state 0x2 ref 0 00000001:00080000:0.0:1789760779.308757:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973520 is committed 00000100:00100000:3.0:1789760779.317271:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.317529:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795712 00000100:00100000:1.0:1789760779.317546:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795712 00000100:00100000:1.0:1789760779.317637:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757674556200 time 757724855640 state 0x0 ref 1 00000100:00100000:1.0:1789760779.317660:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757725116056 time 757724855640 state 0x1 ref 1 00000100:00100000:2.0:1789760779.317825:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757725116056 time 757783109741 state 0x0 ref 1 00000100:00100000:2.0:1789760779.317851:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515795712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.317928:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25411584, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760779.317942:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.317974:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760779.317981:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.317993:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.318015:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.361413:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25411584 -> 25415680 00010000:00010000:3.0:1789760779.361433:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.361445:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.361454:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.361460:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760779.361465:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.361473:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.361490:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.361498:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fc0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.361595:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515795712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43748us (44339us total) trans 4294973521 rc 0/0 00000100:00100000:3.0:1789760779.361606:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6233 00000100:00100000:3.0:1789760779.361616:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757725116056 time 757783109741 state 0x2 ref 0 00000001:00080000:0.0:1789760779.363123:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973521 is committed 00000100:00100000:2.0:1789760779.366139:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.366287:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795840 00000100:00100000:1.0:1789760779.366298:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795840 00000100:00100000:1.0:1789760779.366352:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757725116056 time 757783109741 state 0x0 ref 1 00000100:00100000:1.0:1789760779.366370:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757783370157 time 757783109741 state 0x1 ref 1 00000100:00100000:1.0:1789760779.366441:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757783370157 time 757831729862 state 0x0 ref 1 00000100:00100000:1.0:1789760779.366455:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515795840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.366503:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25415680, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.366512:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.366529:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760779.366534:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.366541:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.366554:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.400892:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25415680 -> 25419776 00000001:00080000:2.0:1789760779.400904:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973522 is committed 00010000:00010000:1.0:1789760779.400911:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.400924:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.400934:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.400942:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.400947:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.400957:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.400976:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.400984:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.401175:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515795840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34713us (35037us total) trans 4294973522 rc 0/0 00000100:00100000:1.0:1789760779.401189:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6234 00000100:00100000:1.0:1789760779.401199:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757783370157 time 757831729862 state 0x2 ref 0 00000100:00100000:2.0:1789760779.408856:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760779.409159:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515795968 00000100:00100000:0.0:1789760779.409178:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515795968 00000100:00100000:0.0:1789760779.409236:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757783370157 time 757831729862 state 0x0 ref 1 00000100:00100000:0.0:1789760779.409251:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757831990278 time 757831729862 state 0x1 ref 1 00000100:00100000:0.0:1789760779.409284:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757831990278 time 757874572557 state 0x0 ref 1 00000100:00100000:0.0:1789760779.409305:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515795968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760779.409351:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25419776, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:0.0:1789760779.409361:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.409378:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760779.409383:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.409390:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.409407:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.432353:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25419776 -> 25423872 00010000:00010000:1.0:1789760779.432373:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.432387:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.432397:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.432405:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.432410:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.432421:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.432441:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.432449:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fc1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.432548:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515795968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23248us (23696us total) trans 4294973523 rc 0/0 00000100:00100000:1.0:1789760779.432559:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6235 00000100:00100000:1.0:1789760779.432569:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757831990278 time 757874572557 state 0x2 ref 0 00000001:00080000:3.0:1789760779.433095:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973523 is committed 00000100:00100000:3.0:1789760779.436132:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.436275:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796096 00000100:00100000:1.0:1789760779.436285:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796096 00000100:00100000:1.0:1789760779.436339:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757831990278 time 757874572557 state 0x0 ref 1 00000100:00100000:1.0:1789760779.436355:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757874832973 time 757874572557 state 0x1 ref 1 00000100:00100000:1.0:1789760779.436422:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757874832973 time 757901710175 state 0x0 ref 1 00000100:00100000:1.0:1789760779.436445:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515796096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.436504:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25423872, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.436514:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.436528:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760779.436533:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.436540:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.436556:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.471156:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25423872 -> 25427968 00010000:00010000:2.0:1789760779.471176:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.471189:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.471198:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.471206:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.471211:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.471219:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.471237:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.471244:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fc23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.471368:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515796096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34932us (35243us total) trans 4294973524 rc 0/0 00000100:00100000:2.0:1789760779.471380:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6236 00000100:00100000:2.0:1789760779.471388:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757874832973 time 757901710175 state 0x2 ref 0 00000001:00080000:3.0:1789760779.471473:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973524 is committed 00000100:00100000:0.0:1789760779.476911:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760779.477012:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796224 00000100:00100000:2.0:1789760779.477022:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796224 00000100:00100000:2.0:1789760779.477070:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757874832973 time 757901710175 state 0x0 ref 1 00000100:00100000:2.0:1789760779.477085:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757901970591 time 757901710175 state 0x1 ref 1 00000100:00100000:2.0:1789760779.477115:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757901970591 time 757942404692 state 0x0 ref 1 00000100:00100000:2.0:1789760779.477128:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515796224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.477173:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25427968, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760779.477181:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.477196:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760779.477200:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.477207:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.477221:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.504559:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25427968 -> 25432064 00000001:00080000:3.0:1789760779.504575:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973525 is committed 00010000:00010000:2.0:1789760779.504579:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.504592:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.504602:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.504609:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.504614:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.504623:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.504642:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.504650:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fc2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.505289:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515796224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28158us (28382us total) trans 4294973525 rc 0/0 00000100:00100000:3.0:1789760779.505301:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6237 00000100:00100000:3.0:1789760779.505310:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757901970591 time 757942404692 state 0x2 ref 0 00000100:00100000:1.0:1789760779.519888:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760779.519986:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796352 00000100:00100000:3.0:1789760779.520005:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796352 00000100:00100000:3.0:1789760779.520083:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757901970591 time 757942404692 state 0x0 ref 1 00000100:00100000:3.0:1789760779.520103:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757942665108 time 757942404692 state 0x1 ref 1 00000100:00100000:3.0:1789760779.520139:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757942665108 time 757985427843 state 0x0 ref 1 00000100:00100000:3.0:1789760779.520157:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760779.520228:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25432064, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:3.0:1789760779.520240:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.520262:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760779.520268:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.520276:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.520295:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.560587:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25432064 -> 25436160 00010000:00010000:2.0:1789760779.560618:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.560635:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.560646:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.560654:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.560660:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.560666:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973526 is committed 00010000:00010000:2.0:1789760779.560670:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.560728:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.560738:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d1fc31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.560848:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40691us (40963us total) trans 4294973526 rc 0/0 00000100:00100000:2.0:1789760779.560859:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6238 00000100:00100000:2.0:1789760779.560871:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757942665108 time 757985427843 state 0x2 ref 0 00000100:00100000:2.0:1789760779.565348:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760779.565477:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796480 00000100:00100000:3.0:1789760779.565491:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796480 00000100:00100000:3.0:1789760779.565559:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757942665108 time 757985427843 state 0x0 ref 1 00000100:00100000:3.0:1789760779.565578:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757985688259 time 757985427843 state 0x1 ref 1 00000100:00100000:3.0:1789760779.565609:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757985688259 time 758030897924 state 0x0 ref 1 00000100:00100000:3.0:1789760779.565622:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760779.565692:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25436160, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:3.0:1789760779.565744:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.565765:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.565770:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.565777:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.565794:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.582118:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760779.596227:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25436160 -> 25440256 00010000:00010000:1.0:1789760779.596256:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596272:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596283:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596292:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.596299:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596309:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596334:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.596424:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973527 is committed 00010000:00010000:1.0:1789760779.596913:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d1fc38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.597043:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31415us (31714us total) trans 4294973527 rc 0/0 00000100:00100000:1.0:1789760779.597053:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6239 00000100:00100000:1.0:1789760779.597064:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757985688259 time 758030897924 state 0x2 ref 0 00000100:00100000:0.0:1789760779.600257:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.600460:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796608 00000100:00100000:1.0:1789760779.600473:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796608 00000100:00100000:1.0:1789760779.600537:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 757985688259 time 758030897924 state 0x0 ref 1 00000100:00100000:1.0:1789760779.600552:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758031158340 time 758030897924 state 0x1 ref 1 00000100:00100000:1.0:1789760779.600577:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758031158340 time 758065867179 state 0x0 ref 1 00000100:00100000:1.0:1789760779.600588:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.600674:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25440256, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.600686:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600801:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.600807:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.600815:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.600833:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.633479:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973528 is committed 00002000:00010000:2.0:1789760779.634406:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25440256 -> 25444352 00010000:00010000:2.0:1789760779.634430:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.634446:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.634456:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.634464:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.634469:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.634481:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.634505:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.634513:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fc3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.634634:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34039us (34384us total) trans 4294973528 rc 0/0 00000100:00100000:2.0:1789760779.634647:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6240 00000100:00100000:2.0:1789760779.634659:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758031158340 time 758065867179 state 0x2 ref 0 00000100:00100000:2.0:1789760779.639346:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.639450:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796736 00000100:00100000:1.0:1789760779.639463:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796736 00000100:00100000:1.0:1789760779.639542:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758031158340 time 758065867179 state 0x0 ref 1 00000100:00100000:1.0:1789760779.639562:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758066127595 time 758065867179 state 0x1 ref 1 00000100:00100000:1.0:1789760779.639630:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758066127595 time 758104918337 state 0x0 ref 1 00000100:00100000:1.0:1789760779.639644:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.639758:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25444352, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:1.0:1789760779.639770:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.639791:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.639796:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.639804:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.639821:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.666745:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973529 is committed 00002000:00010000:2.0:1789760779.666876:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25444352 -> 25448448 00010000:00010000:2.0:1789760779.666897:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.666914:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.666925:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.666940:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.666946:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.666956:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.666982:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.666990:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fc46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.667112:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515796736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27461us (27781us total) trans 4294973529 rc 0/0 00000100:00100000:2.0:1789760779.667124:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6241 00000100:00100000:2.0:1789760779.667136:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758066127595 time 758104918337 state 0x2 ref 0 00000100:00100000:3.0:1789760779.681660:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760779.681852:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796864 00000100:00100000:2.0:1789760779.681866:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796864 00000100:00100000:2.0:1789760779.681939:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758066127595 time 758104918337 state 0x0 ref 1 00000100:00100000:2.0:1789760779.681958:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758105178753 time 758104918337 state 0x1 ref 1 00000100:00100000:2.0:1789760779.681990:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758105178753 time 758147279483 state 0x0 ref 1 00000100:00100000:2.0:1789760779.682008:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515796864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.682070:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25448448, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760779.682078:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682094:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.682098:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.682104:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682116:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.723156:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25448448 -> 25452544 00010000:00010000:2.0:1789760779.723185:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723200:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723211:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723219:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.723225:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723237:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723264:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723273:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.723396:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515796864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41384us (41842us total) trans 4294973530 rc 0/0 00000100:00100000:2.0:1789760779.723409:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6242 00000100:00100000:2.0:1789760779.723422:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758105178753 time 758147279483 state 0x2 ref 0 00000001:00080000:3.0:1789760779.723510:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973530 is committed 00000100:00100000:0.0:1789760779.732587:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760779.733873:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515796992 00000100:00100000:3.0:1789760779.733895:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515796992 00000100:00100000:3.0:1789760779.733977:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758105178753 time 758147279483 state 0x0 ref 1 00000100:00100000:3.0:1789760779.733999:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758147539899 time 758147279483 state 0x1 ref 1 00000100:00100000:3.0:1789760779.734119:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758147539899 time 758199406778 state 0x0 ref 1 00000100:00100000:3.0:1789760779.734135:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515796992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760779.734204:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25452544, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:3.0:1789760779.734215:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.734236:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.734241:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.734248:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.734264:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.774928:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25452544 -> 25456640 00010000:00010000:2.0:1789760779.774958:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.774974:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.774983:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.774992:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760779.774999:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.775009:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.775035:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.775043:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d1fc54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.775166:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515796992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41027us (42588us total) trans 4294973531 rc 0/0 00000100:00100000:2.0:1789760779.775180:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6243 00000100:00100000:2.0:1789760779.775193:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758147539899 time 758199406778 state 0x2 ref 0 00000001:00080000:0.0:1789760779.775275:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973531 is committed 00000100:00100000:0.0:1789760779.776330:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c5f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544419840:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760779.776434:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760779.776607:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419840 00000100:00100000:1.0:1789760779.776628:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419840 00000100:00100000:1.0:1789760779.776737:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:1.0:1789760779.776749:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c224c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544419840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760779.776797:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760779.777105:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c224c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544419840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 358us (670us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760779.777114:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00080000:1.0:1789760779.777403:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760779.777491:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae80540ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544419968:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760779.777527:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.777779:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544419968 00000100:00100000:2.0:1789760779.777787:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544419968 00000100:00100000:2.0:1789760779.777827:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789760779.777836:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544419968:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760779.777855:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526712832 pending 0 free 3832545280 avail 3825205248 00000100:00100000:2.0:1789760779.777896:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544419968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 61us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.777903:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00080000:1.0:1789760779.777962:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760779.778003:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae80540ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544419968:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760779.778094:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c5f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544419840:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760779.784330:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760779.784400:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797120 00000100:00100000:2.0:1789760779.784413:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797120 00000100:00100000:2.0:1789760779.784554:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758147539899 time 758199406778 state 0x0 ref 1 00000100:00100000:2.0:1789760779.784572:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758199667194 time 758199406778 state 0x1 ref 1 00000100:00100000:2.0:1789760779.784609:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758199667194 time 758249897228 state 0x0 ref 1 00000100:00100000:2.0:1789760779.784624:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515797120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.784703:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25456640, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:2.0:1789760779.784746:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.784767:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.784773:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.784781:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.784798:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.818831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973532 is committed 00002000:00010000:3.0:1789760779.822814:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25456640 -> 25460736 00010000:00010000:3.0:1789760779.822843:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.822860:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.822871:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.822879:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760779.822885:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.822894:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.822919:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.822928:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fc5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.823052:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515797120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38424us (38729us total) trans 4294973532 rc 0/0 00000100:00100000:3.0:1789760779.823069:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6244 00000100:00100000:3.0:1789760779.823081:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758199667194 time 758249897228 state 0x2 ref 0 00000100:00100000:3.0:1789760779.825951:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760779.826058:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797248 00000100:00100000:0.0:1789760779.826072:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797248 00000100:00100000:0.0:1789760779.826145:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758199667194 time 758249897228 state 0x0 ref 1 00000100:00100000:0.0:1789760779.826163:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758250157644 time 758249897228 state 0x1 ref 1 00000100:00100000:0.0:1789760779.826258:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758250157644 time 758291545343 state 0x0 ref 1 00000100:00100000:0.0:1789760779.826274:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515797248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760779.826356:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25460736, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc807 00010000:00010000:0.0:1789760779.826367:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.826389:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.826395:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.826403:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.826432:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.851047:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25460736 -> 25464832 00002000:00010000:0.0:1789760779.851061:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:0.0:1789760779.851065:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00010000:00010000:0.0:1789760779.851084:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.851105:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.851117:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.851125:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760779.851131:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.851141:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.851167:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.851175:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fc62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.851237:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973533 is committed 00000100:00100000:0.0:1789760779.851335:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515797248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25057us (25391us total) trans 4294973533 rc 0/0 00000100:00100000:0.0:1789760779.851348:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6245 00000100:00100000:0.0:1789760779.851359:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758250157644 time 758291545343 state 0x2 ref 0 00000100:00100000:0.0:1789760779.855384:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.855955:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797376 00000100:00100000:1.0:1789760779.855970:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797376 00000100:00100000:1.0:1789760779.856047:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758250157644 time 758291545343 state 0x0 ref 1 00000100:00100000:1.0:1789760779.856067:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758291805759 time 758291545343 state 0x1 ref 1 00000100:00100000:1.0:1789760779.856108:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758291805759 time 758321396760 state 0x0 ref 1 00000100:00100000:1.0:1789760779.856123:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515797376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.856416:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25464832, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760779.856424:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.856443:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.856447:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.856452:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.856465:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.888106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973534 is committed 00002000:00010000:1.0:1789760779.888196:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25464832 -> 25468928 00010000:00010000:1.0:1789760779.888217:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888233:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888244:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888252:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.888258:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888269:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888295:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.888304:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.888427:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515797376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32298us (33050us total) trans 4294973534 rc 0/0 00000100:00100000:1.0:1789760779.888439:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6246 00000100:00100000:1.0:1789760779.888452:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758291805759 time 758321396760 state 0x2 ref 0 00000100:00100000:2.0:1789760779.895864:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.895983:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797504 00000100:00100000:1.0:1789760779.895997:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797504 00000100:00100000:1.0:1789760779.896067:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758291805759 time 758321396760 state 0x0 ref 1 00000100:00100000:1.0:1789760779.896085:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758321657176 time 758321396760 state 0x1 ref 1 00000100:00100000:1.0:1789760779.896122:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758321657176 time 758361410725 state 0x0 ref 1 00000100:00100000:1.0:1789760779.897312:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515797504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.897397:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25468928, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760779.897416:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.897440:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.897446:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.897455:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.897478:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.930563:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973535 is committed 00002000:00010000:1.0:1789760779.930833:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25468928 -> 25473024 00010000:00010000:1.0:1789760779.930854:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930872:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930884:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930892:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.930898:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930907:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930932:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930940:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.931082:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515797504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34943us (35228us total) trans 4294973535 rc 0/0 00000100:00100000:1.0:1789760779.931092:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6247 00000100:00100000:1.0:1789760779.931101:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758321657176 time 758361410725 state 0x2 ref 0 00000100:00100000:2.0:1789760779.935275:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760779.935445:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797632 00000100:00100000:1.0:1789760779.935459:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797632 00000100:00100000:1.0:1789760779.935539:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758321657176 time 758361410725 state 0x0 ref 1 00000100:00100000:1.0:1789760779.935561:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758361671141 time 758361410725 state 0x1 ref 1 00000100:00100000:1.0:1789760779.935650:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758361671141 time 758400937583 state 0x0 ref 1 00000100:00100000:1.0:1789760779.935665:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515797632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760779.935759:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25473024, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760779.935770:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.935791:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.935795:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.935804:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.935822:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.969770:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973536 is committed 00002000:00010000:1.0:1789760779.969895:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25473024 -> 25477120 00010000:00010000:1.0:1789760779.969914:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.969929:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.969940:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.969948:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760779.969954:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.969964:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.969990:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.969999:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.970220:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515797632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34549us (34954us total) trans 4294973536 rc 0/0 00000100:00100000:1.0:1789760779.970232:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6248 00000100:00100000:1.0:1789760779.970244:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758361671141 time 758400937583 state 0x2 ref 0 00000100:00100000:0.0:1789760779.973486:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760779.973885:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797760 00000100:00100000:2.0:1789760779.973901:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797760 00000100:00100000:2.0:1789760779.973977:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758361671141 time 758400937583 state 0x0 ref 1 00000100:00100000:2.0:1789760779.973996:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758401197999 time 758400937583 state 0x1 ref 1 00000100:00100000:2.0:1789760779.974057:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758401197999 time 758439345109 state 0x0 ref 1 00000100:00100000:2.0:1789760779.974072:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515797760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760779.974216:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25477120, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:2.0:1789760779.974227:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.974246:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.974318:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.974328:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.974347:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.005075:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973537 is committed 00002000:00010000:2.0:1789760780.005393:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25477120 -> 25481216 00010000:00010000:2.0:1789760780.005510:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005526:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005537:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005546:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760780.005553:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005564:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005589:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.005598:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fc7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.005926:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515797760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31849us (32446us total) trans 4294973537 rc 0/0 00000100:00100000:2.0:1789760780.005938:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6249 00000100:00100000:2.0:1789760780.005950:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758401197999 time 758439345109 state 0x2 ref 0 00000100:00100000:2.0:1789760780.008536:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.008937:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515797888 00000100:00100000:3.0:1789760780.008952:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515797888 00000100:00100000:3.0:1789760780.009040:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758401197999 time 758439345109 state 0x0 ref 1 00000100:00100000:3.0:1789760780.009055:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758439605525 time 758439345109 state 0x1 ref 1 00000100:00100000:3.0:1789760780.009088:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758439605525 time 758474376027 state 0x0 ref 1 00000100:00100000:3.0:1789760780.009103:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515797888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.009174:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25481216, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:3.0:1789760780.009185:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.009205:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760780.009209:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.009217:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.009232:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760780.009369:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760780.046115:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25481216 -> 25485312 00010000:00010000:3.0:1789760780.046140:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.046152:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.046159:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.046164:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.046169:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.046176:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.046247:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.046259:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fc85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.046854:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515797888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37747us (38337us total) trans 4294973538 rc 0/0 00000100:00100000:3.0:1789760780.046880:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6250 00000100:00100000:3.0:1789760780.046893:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758439605525 time 758474376027 state 0x2 ref 0 00000001:00080000:3.0:1789760780.046948:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973538 is committed 00000100:00100000:3.0:1789760780.052181:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760780.052279:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798016 00000100:00100000:0.0:1789760780.052292:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798016 00000100:00100000:0.0:1789760780.052361:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758439605525 time 758474376027 state 0x0 ref 1 00000100:00100000:0.0:1789760780.052381:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758474636443 time 758474376027 state 0x1 ref 1 00000100:00100000:0.0:1789760780.052478:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758474636443 time 758517766754 state 0x0 ref 1 00000100:00100000:0.0:1789760780.052494:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515798016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760780.052561:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25485312, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:0.0:1789760780.052574:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.052596:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760780.052601:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.052609:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.052627:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.086535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973539 is committed 00002000:00010000:3.0:1789760780.086794:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25485312 -> 25489408 00010000:00010000:3.0:1789760780.086814:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.086841:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.086852:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.086861:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.086867:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.086878:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.086903:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.086911:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fc8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.087036:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515798016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34537us (34871us total) trans 4294973539 rc 0/0 00000100:00100000:3.0:1789760780.087048:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6251 00000100:00100000:3.0:1789760780.087061:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758474636443 time 758517766754 state 0x2 ref 0 00000100:00100000:0.0:1789760780.096332:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.096462:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798144 00000100:00100000:1.0:1789760780.096476:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798144 00000100:00100000:1.0:1789760780.096551:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758474636443 time 758517766754 state 0x0 ref 1 00000100:00100000:1.0:1789760780.096570:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758518027170 time 758517766754 state 0x1 ref 1 00000100:00100000:1.0:1789760780.096608:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758518027170 time 758561896749 state 0x0 ref 1 00000100:00100000:1.0:1789760780.096627:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.096748:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25489408, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.096759:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.096779:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760780.096784:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.096793:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.096810:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.122037:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973540 is committed 00002000:00010000:3.0:1789760780.122340:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25489408 -> 25493504 00010000:00010000:3.0:1789760780.122358:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.122370:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.122377:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.122382:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.122387:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.122394:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.122414:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.122424:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fc93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.122579:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25952us (26254us total) trans 4294973540 rc 0/0 00000100:00100000:3.0:1789760780.122591:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6252 00000100:00100000:3.0:1789760780.122604:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758518027170 time 758561896749 state 0x2 ref 0 00000100:00100000:2.0:1789760780.126311:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.126493:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798272 00000100:00100000:1.0:1789760780.126507:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798272 00000100:00100000:1.0:1789760780.126816:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758518027170 time 758561896749 state 0x0 ref 1 00000100:00100000:1.0:1789760780.126836:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758562157165 time 758561896749 state 0x1 ref 1 00000100:00100000:1.0:1789760780.126959:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758562157165 time 758592246375 state 0x0 ref 1 00000100:00100000:1.0:1789760780.126975:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515798272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.127039:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25493504, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.127050:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.127071:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760780.127077:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.127085:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.127101:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.176658:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973541 is committed 00002000:00010000:2.0:1789760780.176677:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25493504 -> 25497600 00010000:00010000:2.0:1789760780.176904:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.176922:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.176938:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.176947:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760780.176953:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.176965:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.177249:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.177261:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1fc9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.177530:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515798272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50552us (51226us total) trans 4294973541 rc 0/0 00000100:00100000:2.0:1789760780.177543:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6253 00000100:00100000:2.0:1789760780.177557:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758562157165 time 758592246375 state 0x2 ref 0 00000100:00100000:2.0:1789760780.181978:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.182055:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798400 00000100:00100000:1.0:1789760780.182068:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798400 00000100:00100000:1.0:1789760780.182140:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758562157165 time 758592246375 state 0x0 ref 1 00000100:00100000:1.0:1789760780.182159:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758592506791 time 758592246375 state 0x1 ref 1 00000100:00100000:1.0:1789760780.182193:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758592506791 time 758647481219 state 0x0 ref 1 00000100:00100000:1.0:1789760780.182206:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.182281:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25497600, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.182292:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.182315:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760780.182321:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.182328:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.182345:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.223450:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973542 is committed 00002000:00010000:3.0:1789760780.223603:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25497600 -> 25501696 00010000:00010000:3.0:1789760780.223634:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223651:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223661:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223669:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.223674:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223684:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223764:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223773:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fca1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.223897:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41686us (41927us total) trans 4294973542 rc 0/0 00000100:00100000:3.0:1789760780.223910:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6254 00000100:00100000:3.0:1789760780.223923:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758592506791 time 758647481219 state 0x2 ref 0 00000100:00100000:0.0:1789760780.227065:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760780.227177:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798528 00000100:00100000:2.0:1789760780.227190:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798528 00000100:00100000:2.0:1789760780.227264:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758592506791 time 758647481219 state 0x0 ref 1 00000100:00100000:2.0:1789760780.227286:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758647741635 time 758647481219 state 0x1 ref 1 00000100:00100000:2.0:1789760780.227319:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758647741635 time 758692608318 state 0x0 ref 1 00000100:00100000:2.0:1789760780.227333:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515798528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760780.227412:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25501696, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:2.0:1789760780.227424:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.227446:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760780.227451:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.227458:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.227476:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.261428:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973543 is committed 00002000:00010000:3.0:1789760780.261586:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25501696 -> 25505792 00010000:00010000:3.0:1789760780.261608:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.261623:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.261632:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.261640:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.261645:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.261661:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.261687:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.261738:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fca8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.261870:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515798528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34532us (34813us total) trans 4294973543 rc 0/0 00000100:00100000:3.0:1789760780.261882:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6255 00000100:00100000:3.0:1789760780.261894:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758647741635 time 758692608318 state 0x2 ref 0 00000100:00100000:2.0:1789760780.265661:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.265882:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798656 00000100:00100000:3.0:1789760780.265896:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798656 00000100:00100000:3.0:1789760780.265972:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758647741635 time 758692608318 state 0x0 ref 1 00000100:00100000:3.0:1789760780.265991:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758692868734 time 758692608318 state 0x1 ref 1 00000100:00100000:3.0:1789760780.266063:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758692868734 time 758731350376 state 0x0 ref 1 00000100:00100000:3.0:1789760780.266079:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.266148:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25505792, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:3.0:1789760780.266159:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.266180:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760780.266186:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.266194:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.266211:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.297989:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973544 is committed 00002000:00010000:0.0:1789760780.298110:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25505792 -> 25509888 00010000:00010000:0.0:1789760780.298133:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298148:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298158:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298167:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760780.298175:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298186:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298211:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.298220:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d1fcaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.298398:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32314us (32743us total) trans 4294973544 rc 0/0 00000100:00100000:0.0:1789760780.298410:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6256 00000100:00100000:0.0:1789760780.298423:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758692868734 time 758731350376 state 0x2 ref 0 00000100:00100000:2.0:1789760780.302307:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.302409:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798784 00000100:00100000:3.0:1789760780.302423:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798784 00000100:00100000:3.0:1789760780.302505:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758692868734 time 758731350376 state 0x0 ref 1 00000100:00100000:3.0:1789760780.302527:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758731610792 time 758731350376 state 0x1 ref 1 00000100:00100000:3.0:1789760780.302567:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758731610792 time 758767855881 state 0x0 ref 1 00000100:00100000:3.0:1789760780.302585:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.302663:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25509888, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:3.0:1789760780.302681:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.302745:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760780.302750:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.302760:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.302783:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.345077:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25509888 -> 25513984 00010000:00010000:0.0:1789760780.345108:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.345125:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.345136:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.345144:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760780.345150:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.345161:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.345187:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.345195:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d1fcb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.345319:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515798784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42728us (43019us total) trans 4294973545 rc 0/0 00000100:00100000:0.0:1789760780.345332:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6257 00000100:00100000:0.0:1789760780.345358:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758731610792 time 758767855881 state 0x2 ref 0 00000001:00080000:3.0:1789760780.346287:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973545 is committed 00000100:00100000:2.0:1789760780.349332:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.349421:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515798912 00000100:00100000:3.0:1789760780.349437:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515798912 00000100:00100000:3.0:1789760780.349507:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758731610792 time 758767855881 state 0x0 ref 1 00000100:00100000:3.0:1789760780.349524:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758768116297 time 758767855881 state 0x1 ref 1 00000100:00100000:3.0:1789760780.349557:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758768116297 time 758814847001 state 0x0 ref 1 00000100:00100000:3.0:1789760780.349569:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.349642:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25513984, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:3.0:1789760780.349653:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.349672:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760780.349677:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.349685:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.349756:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.402140:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973546 is committed 00002000:00010000:3.0:1789760780.402504:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25513984 -> 25518080 00010000:00010000:3.0:1789760780.402526:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.402544:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.402555:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.402563:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.402569:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.402579:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.402602:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.402610:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fcbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.402788:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515798912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53212us (53464us total) trans 4294973546 rc 0/0 00000100:00100000:3.0:1789760780.402804:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6258 00000100:00100000:3.0:1789760780.402816:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758768116297 time 758814847001 state 0x2 ref 0 00000100:00100000:2.0:1789760780.408313:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.408407:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799040 00000100:00100000:1.0:1789760780.408420:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799040 00000100:00100000:1.0:1789760780.408493:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758768116297 time 758814847001 state 0x0 ref 1 00000100:00100000:1.0:1789760780.408522:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758815107417 time 758814847001 state 0x1 ref 1 00000100:00100000:1.0:1789760780.408558:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758815107417 time 758873846433 state 0x0 ref 1 00000100:00100000:1.0:1789760780.408572:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.408637:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25518080, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.408648:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.408668:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760780.408674:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.408682:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.408747:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760780.452587:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25518080 -> 25522176 00010000:00010000:1.0:1789760780.452680:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.452734:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.452744:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.452749:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973547 is committed 00010000:00010000:1.0:1789760780.452751:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760780.452757:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.452765:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.453131:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.453141:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fcc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.453441:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44864us (45135us total) trans 4294973547 rc 0/0 00000100:00100000:1.0:1789760780.453464:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6259 00000100:00100000:1.0:1789760780.453477:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758815107417 time 758873846433 state 0x2 ref 0 00000100:00100000:0.0:1789760780.459560:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.459673:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799168 00000100:00100000:1.0:1789760780.459686:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799168 00000100:00100000:1.0:1789760780.459795:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758815107417 time 758873846433 state 0x0 ref 1 00000100:00100000:1.0:1789760780.459816:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758874106849 time 758873846433 state 0x1 ref 1 00000100:00100000:1.0:1789760780.459841:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758874106849 time 758925129897 state 0x0 ref 1 00000100:00100000:1.0:1789760780.459855:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515799168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.459921:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25522176, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.459933:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.459952:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760780.459956:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.459964:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.459980:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.503408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973548 is committed 00002000:00010000:3.0:1789760780.503602:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25522176 -> 25526272 00010000:00010000:3.0:1789760780.503634:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.503650:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.503661:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.503669:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.503675:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.503684:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.503756:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.503766:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fccb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.503891:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515799168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44031us (44337us total) trans 4294973548 rc 0/0 00000100:00100000:3.0:1789760780.503904:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6260 00000100:00100000:3.0:1789760780.503917:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758874106849 time 758925129897 state 0x2 ref 0 00000100:00100000:2.0:1789760780.513009:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760780.514267:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799296 00000100:00100000:0.0:1789760780.514285:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799296 00000100:00100000:0.0:1789760780.514376:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758874106849 time 758925129897 state 0x0 ref 1 00000100:00100000:0.0:1789760780.514396:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758925390313 time 758925129897 state 0x1 ref 1 00000100:00100000:0.0:1789760780.514433:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758925390313 time 758979722008 state 0x0 ref 1 00000100:00100000:0.0:1789760780.514450:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760780.514518:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25526272, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:0.0:1789760780.514529:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.514548:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760780.514554:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.514634:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.514654:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.559033:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973549 is committed 00002000:00010000:0.0:1789760780.559195:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25526272 -> 25530368 00010000:00010000:0.0:1789760780.559216:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.559231:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.559241:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.559249:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760780.559255:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.559265:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.559293:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.559302:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fcd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.559426:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44972us (46423us total) trans 4294973549 rc 0/0 00000100:00100000:0.0:1789760780.559437:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6261 00000100:00100000:0.0:1789760780.559453:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758925390313 time 758979722008 state 0x2 ref 0 00000100:00100000:2.0:1789760780.562284:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760780.562335:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799424 00000100:00100000:0.0:1789760780.562348:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799424 00000100:00100000:0.0:1789760780.562426:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758925390313 time 758979722008 state 0x0 ref 1 00000100:00100000:0.0:1789760780.562454:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758979982424 time 758979722008 state 0x1 ref 1 00000100:00100000:0.0:1789760780.562493:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758979982424 time 759027780960 state 0x0 ref 1 00000100:00100000:0.0:1789760780.562507:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515799424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760780.562571:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25530368, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:0.0:1789760780.562582:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.562602:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.562607:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.562614:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.562640:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.602146:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25530368 -> 25534464 00010000:00010000:2.0:1789760780.602175:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.602191:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.602202:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.602210:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760780.602216:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.602227:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.602250:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.602258:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fcd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.602391:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515799424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39879us (40112us total) trans 4294973550 rc 0/0 00000100:00100000:2.0:1789760780.602404:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6262 00000100:00100000:2.0:1789760780.602416:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758979982424 time 759027780960 state 0x2 ref 0 00000001:00080000:2.0:1789760780.602480:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973550 is committed 00010000:00010000:0.0:1789760780.606176:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760780.608967:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.609155:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799552 00000100:00100000:3.0:1789760780.609169:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799552 00000100:00100000:3.0:1789760780.609247:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 758979982424 time 759027780960 state 0x0 ref 1 00000100:00100000:3.0:1789760780.609267:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759028041376 time 759027780960 state 0x1 ref 1 00000100:00100000:3.0:1789760780.609354:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759028041376 time 759074642412 state 0x0 ref 1 00000100:00100000:3.0:1789760780.609370:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.609442:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25534464, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:3.0:1789760780.609573:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.609596:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.609600:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.609616:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.609633:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.659556:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973551 is committed 00002000:00010000:3.0:1789760780.659852:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25534464 -> 25538560 00010000:00010000:3.0:1789760780.659873:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.659889:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.659900:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.659909:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760780.659915:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.659925:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.659951:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.659960:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.660114:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50740us (51157us total) trans 4294973551 rc 0/0 00000100:00100000:3.0:1789760780.660131:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6263 00000100:00100000:3.0:1789760780.660144:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759028041376 time 759074642412 state 0x2 ref 0 00000100:00100000:0.0:1789760780.663994:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.666935:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799680 00000100:00100000:3.0:1789760780.666961:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799680 00000100:00100000:3.0:1789760780.667062:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759028041376 time 759074642412 state 0x0 ref 1 00000100:00100000:3.0:1789760780.667082:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759074902828 time 759074642412 state 0x1 ref 1 00000100:00100000:3.0:1789760780.667123:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759074902828 time 759132411266 state 0x0 ref 1 00000100:00100000:3.0:1789760780.667139:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.667201:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25538560, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:3.0:1789760780.667211:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.667233:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.667238:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.667246:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.667263:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.702962:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973552 is committed 00002000:00010000:1.0:1789760780.703055:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25538560 -> 25542656 00010000:00010000:1.0:1789760780.703077:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.703093:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.703111:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.703120:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760780.703126:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.703137:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.703164:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.703172:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d1fce7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.703301:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515799680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36158us (39315us total) trans 4294973552 rc 0/0 00000100:00100000:1.0:1789760780.703312:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6264 00000100:00100000:1.0:1789760780.703324:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759074902828 time 759132411266 state 0x2 ref 0 00000100:00100000:2.0:1789760780.707990:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.708323:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799808 00000100:00100000:1.0:1789760780.708337:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799808 00000100:00100000:1.0:1789760780.708409:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759074902828 time 759132411266 state 0x0 ref 1 00000100:00100000:1.0:1789760780.708427:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759132671682 time 759132411266 state 0x1 ref 1 00000100:00100000:1.0:1789760780.708619:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759132671682 time 759173905513 state 0x0 ref 1 00000100:00100000:1.0:1789760780.708635:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515799808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.708904:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25542656, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.708916:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.708936:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.708941:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.708950:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.708968:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.746955:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973553 is committed 00002000:00010000:1.0:1789760780.747180:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25542656 -> 25546752 00010000:00010000:1.0:1789760780.747202:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.747217:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.747228:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.747237:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760780.747244:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.747254:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.747278:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.747286:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fcee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.747451:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515799808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38813us (39468us total) trans 4294973553 rc 0/0 00000100:00100000:1.0:1789760780.747464:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6265 00000100:00100000:1.0:1789760780.747476:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759132671682 time 759173905513 state 0x2 ref 0 00000100:00100000:2.0:1789760780.750480:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.750585:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515799936 00000100:00100000:1.0:1789760780.750600:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515799936 00000100:00100000:1.0:1789760780.750676:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759132671682 time 759173905513 state 0x0 ref 1 00000100:00100000:1.0:1789760780.750744:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759174165929 time 759173905513 state 0x1 ref 1 00000100:00100000:1.0:1789760780.751039:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759174165929 time 759216325486 state 0x0 ref 1 00000100:00100000:1.0:1789760780.751057:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515799936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.751128:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25546752, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:1.0:1789760780.751140:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.751160:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.751166:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.751172:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.751188:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.788431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973554 is committed 00002000:00010000:2.0:1789760780.788949:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25546752 -> 25550848 00010000:00010000:2.0:1789760780.788970:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.788983:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.788991:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.788998:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760780.789004:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.789014:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.789037:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.789046:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fcf5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.789371:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515799936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38311us (38897us total) trans 4294973554 rc 0/0 00000100:00100000:2.0:1789760780.789382:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6266 00000100:00100000:2.0:1789760780.789392:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759174165929 time 759216325486 state 0x2 ref 0 00000100:00100000:0.0:1789760780.795303:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760780.795389:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800064 00000100:00100000:2.0:1789760780.795406:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800064 00000100:00100000:2.0:1789760780.795493:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759174165929 time 759216325486 state 0x0 ref 1 00000100:00100000:2.0:1789760780.795513:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759216585902 time 759216325486 state 0x1 ref 1 00000100:00100000:2.0:1789760780.795552:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759216585902 time 759260840651 state 0x0 ref 1 00000100:00100000:2.0:1789760780.795569:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515800064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760780.795640:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25550848, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:2.0:1789760780.795653:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.795676:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.795682:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.795700:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.796018:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.827363:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973555 is committed 00002000:00010000:2.0:1789760780.827549:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25550848 -> 25554944 00010000:00010000:2.0:1789760780.827564:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.827577:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.827586:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.827592:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760780.827597:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.827606:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.827625:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.827632:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fcfc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.827765:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515800064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32196us (32468us total) trans 4294973555 rc 0/0 00000100:00100000:2.0:1789760780.827778:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6267 00000100:00100000:2.0:1789760780.827788:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759216585902 time 759260840651 state 0x2 ref 0 00000100:00100000:0.0:1789760780.837862:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760780.838088:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800192 00000100:00100000:2.0:1789760780.838097:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800192 00000100:00100000:2.0:1789760780.838150:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759216585902 time 759260840651 state 0x0 ref 1 00000100:00100000:2.0:1789760780.838165:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759261101067 time 759260840651 state 0x1 ref 1 00000100:00100000:2.0:1789760780.838192:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759261101067 time 759303481759 state 0x0 ref 1 00000100:00100000:2.0:1789760780.838203:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515800192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760780.838247:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25554944, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc807 00010000:00010000:2.0:1789760780.838255:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.838270:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.838275:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.838281:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.838294:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.870305:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973556 is committed 00002000:00010000:2.0:1789760780.870343:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25554944 -> 25559040 00002000:00010000:2.0:1789760780.870348:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:2.0:1789760780.870351:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00010000:00010000:2.0:1789760780.870364:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870377:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870386:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870392:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760780.870398:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.870407:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.871547:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.871563:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.871746:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515800192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33538us (33886us total) trans 4294973556 rc 0/0 00000100:00100000:2.0:1789760780.871880:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6268 00000100:00100000:2.0:1789760780.871890:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759261101067 time 759303481759 state 0x2 ref 0 00000100:00100000:0.0:1789760780.877585:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760780.877659:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800320 00000100:00100000:3.0:1789760780.877669:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800320 00000100:00100000:3.0:1789760780.877826:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759261101067 time 759303481759 state 0x0 ref 1 00000100:00100000:3.0:1789760780.877842:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759303742175 time 759303481759 state 0x1 ref 1 00000100:00100000:3.0:1789760780.877874:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759303742175 time 759343163148 state 0x0 ref 1 00000100:00100000:3.0:1789760780.877886:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515800320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.877937:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25559040, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760780.877948:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.877964:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.877969:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.877975:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.877990:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.920393:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25559040 -> 25563136 00010000:00010000:0.0:1789760780.920414:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.920436:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.920444:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.920451:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760780.920456:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.920465:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.920485:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.920492:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d1fd0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.920586:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515800320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42697us (43005us total) trans 4294973557 rc 0/0 00000100:00100000:0.0:1789760780.920596:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6269 00000100:00100000:0.0:1789760780.920605:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759303742175 time 759343163148 state 0x2 ref 0 00000001:00080000:2.0:1789760780.920964:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973557 is committed 00000100:00100000:0.0:1789760780.927543:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.927646:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800448 00000100:00100000:1.0:1789760780.927657:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800448 00000100:00100000:1.0:1789760780.927743:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759303742175 time 759343163148 state 0x0 ref 1 00000100:00100000:1.0:1789760780.927758:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759343423564 time 759343163148 state 0x1 ref 1 00000100:00100000:1.0:1789760780.927791:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759343423564 time 759393079811 state 0x0 ref 1 00000100:00100000:3.0:1789760780.927868:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515800448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760780.927918:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25563136, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760780.927927:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927946:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.927951:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.927957:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927970:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.954343:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973558 is committed 00002000:00010000:1.0:1789760780.954433:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25563136 -> 25567232 00010000:00010000:1.0:1789760780.954450:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.954463:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.954472:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.954484:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760780.954489:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.954498:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.954516:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.954523:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fd11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.954613:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515800448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26808us (27073us total) trans 4294973558 rc 0/0 00000100:00100000:1.0:1789760780.954624:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6270 00000100:00100000:1.0:1789760780.954633:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759343423564 time 759393079811 state 0x2 ref 0 00000100:00100000:0.0:1789760780.960942:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760780.961007:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800576 00000100:00100000:1.0:1789760780.961016:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800576 00000100:00100000:1.0:1789760780.961064:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759343423564 time 759393079811 state 0x0 ref 1 00000100:00100000:1.0:1789760780.961080:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759393340227 time 759393079811 state 0x1 ref 1 00000100:00100000:1.0:1789760780.961255:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759393340227 time 759426543943 state 0x0 ref 1 00000100:00100000:1.0:1789760780.961268:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515800576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760780.961309:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25567232, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:1.0:1789760780.961316:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.961330:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.961333:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.961337:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.961347:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.003467:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973559 is committed 00002000:00010000:2.0:1789760781.005620:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25567232 -> 25571328 00010000:00010000:2.0:1789760781.005639:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.005652:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.005662:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.005669:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760781.005674:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.005683:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.005732:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.005741:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fd18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.005839:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515800576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44567us (45164us total) trans 4294973559 rc 0/0 00000100:00100000:2.0:1789760781.005852:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6271 00000100:00100000:2.0:1789760781.005861:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759393340227 time 759426543943 state 0x2 ref 0 00000100:00100000:0.0:1789760781.011640:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760781.011905:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800704 00000100:00100000:2.0:1789760781.011915:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800704 00000100:00100000:2.0:1789760781.011972:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759393340227 time 759426543943 state 0x0 ref 1 00000100:00100000:2.0:1789760781.012129:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759426804359 time 759426543943 state 0x1 ref 1 00000100:00100000:2.0:1789760781.012201:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759426804359 time 759477488419 state 0x0 ref 1 00000100:00100000:2.0:1789760781.012215:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515800704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760781.012262:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25571328, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:2.0:1789760781.012273:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.012289:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760781.012293:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.012300:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.012313:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.058095:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973560 is committed 00002000:00010000:3.0:1789760781.058591:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25571328 -> 25575424 00010000:00010000:3.0:1789760781.058607:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.058621:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.058629:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.058788:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.058793:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.058804:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.058830:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.058838:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fd1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.058943:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515800704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46724us (47307us total) trans 4294973560 rc 0/0 00000100:00100000:3.0:1789760781.059128:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6272 00000100:00100000:3.0:1789760781.059138:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759426804359 time 759477488419 state 0x2 ref 0 00000100:00100000:0.0:1789760781.068207:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760781.068305:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800832 00000100:00100000:2.0:1789760781.068315:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800832 00000100:00100000:2.0:1789760781.068370:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759426804359 time 759477488419 state 0x0 ref 1 00000100:00100000:2.0:1789760781.068386:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759477748835 time 759477488419 state 0x1 ref 1 00000100:00100000:2.0:1789760781.068469:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759477748835 time 759533757253 state 0x0 ref 1 00000100:00100000:2.0:1789760781.068481:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515800832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760781.068526:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25575424, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:2.0:1789760781.068535:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.068550:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760781.068554:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.068560:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.068573:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.099454:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25575424 -> 25579520 00010000:00010000:3.0:1789760781.099476:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.099490:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.099500:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.099507:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.099512:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.099527:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.099547:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.099555:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fd26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.099665:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515800832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31180us (31460us total) trans 4294973561 rc 0/0 00000100:00100000:3.0:1789760781.099675:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6273 00000100:00100000:3.0:1789760781.099684:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759477748835 time 759533757253 state 0x2 ref 0 00000001:00080000:0.0:1789760781.100224:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973561 is committed 00000100:00100000:0.0:1789760781.106655:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760781.106776:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515800960 00000100:00100000:2.0:1789760781.106787:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515800960 00000100:00100000:2.0:1789760781.106845:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759477748835 time 759533757253 state 0x0 ref 1 00000100:00100000:2.0:1789760781.106861:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759534017669 time 759533757253 state 0x1 ref 1 00000100:00100000:2.0:1789760781.106899:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759534017669 time 759572188704 state 0x0 ref 1 00000100:00100000:2.0:1789760781.106912:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515800960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760781.106963:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25579520, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:2.0:1789760781.106972:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.107300:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760781.107305:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.107312:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.107327:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.137845:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973562 is committed 00002000:00010000:3.0:1789760781.137994:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25579520 -> 25583616 00010000:00010000:3.0:1789760781.138009:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.138022:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.138029:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.138036:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.138041:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.138151:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.138172:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.138179:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fd2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.138291:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515800960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31377us (31640us total) trans 4294973562 rc 0/0 00000100:00100000:3.0:1789760781.138300:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6274 00000100:00100000:3.0:1789760781.138309:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759534017669 time 759572188704 state 0x2 ref 0 00000100:00100000:2.0:1789760781.142979:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760781.143035:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801088 00000100:00100000:0.0:1789760781.143045:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801088 00000100:00100000:0.0:1789760781.143117:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759534017669 time 759572188704 state 0x0 ref 1 00000100:00100000:0.0:1789760781.143132:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759572449120 time 759572188704 state 0x1 ref 1 00000100:00100000:0.0:1789760781.143164:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759572449120 time 759608452649 state 0x0 ref 1 00000100:00100000:0.0:1789760781.143181:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760781.143245:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25583616, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:0.0:1789760781.143254:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.143269:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760781.143273:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.143278:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.143297:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.178832:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973563 is committed 00002000:00010000:3.0:1789760781.178932:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25583616 -> 25587712 00010000:00010000:3.0:1789760781.178946:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.178958:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.178967:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.178979:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.178984:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.178993:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.179013:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.179021:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d1fd34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.179254:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36072us (36278us total) trans 4294973563 rc 0/0 00000100:00100000:3.0:1789760781.179265:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6275 00000100:00100000:3.0:1789760781.179280:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759572449120 time 759608452649 state 0x2 ref 0 00000100:00100000:0.0:1789760781.182478:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760781.182548:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801216 00000100:00100000:1.0:1789760781.182558:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801216 00000100:00100000:1.0:1789760781.182612:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759572449120 time 759608452649 state 0x0 ref 1 00000100:00100000:1.0:1789760781.182627:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759608713065 time 759608452649 state 0x1 ref 1 00000100:00100000:1.0:1789760781.182659:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759608713065 time 759647948215 state 0x0 ref 1 00000100:00100000:1.0:1789760781.182672:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760781.182760:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25587712, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:1.0:1789760781.182768:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.182781:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760781.182784:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.182789:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.182799:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.211155:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973564 is committed 00002000:00010000:0.0:1789760781.211349:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25587712 -> 25591808 00010000:00010000:0.0:1789760781.211366:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.211379:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.211387:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.211394:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760781.211399:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.211407:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.211426:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.211434:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fd3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.211528:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28854us (29054us total) trans 4294973564 rc 0/0 00000100:00100000:0.0:1789760781.211539:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6276 00000100:00100000:0.0:1789760781.211551:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759608713065 time 759647948215 state 0x2 ref 0 00000100:00100000:0.0:1789760781.218190:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760781.218259:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801344 00000100:00100000:1.0:1789760781.218269:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801344 00000100:00100000:1.0:1789760781.218317:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759608713065 time 759647948215 state 0x0 ref 1 00000100:00100000:1.0:1789760781.218428:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759648208631 time 759647948215 state 0x1 ref 1 00000100:00100000:1.0:1789760781.218500:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759648208631 time 759683787702 state 0x0 ref 1 00000100:00100000:1.0:1789760781.218513:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515801344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760781.218561:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25591808, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:1.0:1789760781.218571:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.218591:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760781.218595:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.218602:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.218669:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.258886:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973565 is committed 00002000:00010000:1.0:1789760781.259013:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25591808 -> 25595904 00010000:00010000:1.0:1789760781.259028:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.259040:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.259050:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.259058:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760781.259063:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.259072:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.259093:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.259101:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.259310:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515801344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40795us (41130us total) trans 4294973565 rc 0/0 00000100:00100000:1.0:1789760781.259321:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6277 00000100:00100000:1.0:1789760781.259331:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759648208631 time 759683787702 state 0x2 ref 0 00000100:00100000:2.0:1789760781.263523:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.263587:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801472 00000100:00100000:3.0:1789760781.263599:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801472 00000100:00100000:3.0:1789760781.263659:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759648208631 time 759683787702 state 0x0 ref 1 00000100:00100000:3.0:1789760781.263674:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759684048118 time 759683787702 state 0x1 ref 1 00000100:00100000:3.0:1789760781.263743:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759684048118 time 759729031156 state 0x0 ref 1 00000100:00100000:3.0:1789760781.263766:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.263815:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25595904, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.263825:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.263841:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760781.263846:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.263853:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.263866:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.322512:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25595904 -> 25600000 00010000:00010000:3.0:1789760781.322532:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.322544:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.322553:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.322560:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.322566:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.322575:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.322595:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.322762:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fd49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.322865:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515801472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59097us (59345us total) trans 4294973566 rc 0/0 00000100:00100000:3.0:1789760781.322878:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6278 00000100:00100000:3.0:1789760781.322889:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759684048118 time 759729031156 state 0x2 ref 0 00000001:00080000:1.0:1789760781.323409:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973566 is committed 00000100:00100000:0.0:1789760781.326044:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.326184:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801600 00000100:00100000:3.0:1789760781.326194:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801600 00000100:00100000:3.0:1789760781.326303:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759684048118 time 759729031156 state 0x0 ref 1 00000100:00100000:3.0:1789760781.326321:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759729291572 time 759729031156 state 0x1 ref 1 00000100:00100000:3.0:1789760781.326354:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759729291572 time 759791643041 state 0x0 ref 1 00000100:00100000:3.0:1789760781.326366:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806249c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.326412:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25600000, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.326421:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.326436:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760781.326441:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.326447:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.326459:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.363858:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973567 is committed 00002000:00010000:3.0:1789760781.364876:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25600000 -> 25604096 00010000:00010000:3.0:1789760781.364895:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.364908:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.364917:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.364924:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.364929:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.364939:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.364964:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.364971:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.365072:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806249c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38702us (39031us total) trans 4294973567 rc 0/0 00000100:00100000:3.0:1789760781.365082:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6279 00000100:00100000:3.0:1789760781.365094:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759729291572 time 759791643041 state 0x2 ref 0 00000100:00100000:2.0:1789760781.372019:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.372108:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801728 00000100:00100000:3.0:1789760781.372117:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801728 00000100:00100000:3.0:1789760781.372171:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759729291572 time 759791643041 state 0x0 ref 1 00000100:00100000:3.0:1789760781.372187:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759791903457 time 759791643041 state 0x1 ref 1 00000100:00100000:3.0:1789760781.372220:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759791903457 time 759837509097 state 0x0 ref 1 00000100:00100000:3.0:1789760781.372233:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.372413:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25604096, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.372424:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.372439:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760781.372445:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.372452:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.372467:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.435043:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973568 is committed 00002000:00010000:3.0:1789760781.435120:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25604096 -> 25608192 00010000:00010000:3.0:1789760781.435137:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.435148:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.435158:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.435164:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.435169:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.435178:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.435198:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.435214:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.435313:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c066680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63076us (63301us total) trans 4294973568 rc 0/0 00000100:00100000:3.0:1789760781.435323:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6280 00000100:00100000:3.0:1789760781.435334:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759791903457 time 759837509097 state 0x2 ref 0 00000100:00100000:2.0:1789760781.440404:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760781.441637:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801856 00000100:00100000:0.0:1789760781.441648:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801856 00000100:00100000:0.0:1789760781.441736:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759791903457 time 759837509097 state 0x0 ref 1 00000100:00100000:0.0:1789760781.441752:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759837769513 time 759837509097 state 0x1 ref 1 00000100:00100000:0.0:1789760781.441778:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759837769513 time 759907067229 state 0x0 ref 1 00000100:00100000:0.0:1789760781.441791:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515801856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760781.441838:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25608192, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:0.0:1789760781.441847:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.441864:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760781.441868:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.441875:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.441893:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760781.487075:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25608192 -> 25612288 00010000:00010000:0.0:1789760781.487095:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.487107:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.487116:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.487123:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760781.487128:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.487138:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.487163:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.487171:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1fd5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.487363:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515801856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45570us (46964us total) trans 4294973569 rc 0/0 00000100:00100000:0.0:1789760781.487375:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6281 00000100:00100000:0.0:1789760781.487384:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759837769513 time 759907067229 state 0x2 ref 0 00000001:00080000:1.0:1789760781.487658:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973569 is committed 00000100:00100000:0.0:1789760781.493923:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760781.493982:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515801984 00000100:00100000:1.0:1789760781.493993:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515801984 00000100:00100000:1.0:1789760781.494048:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759837769513 time 759907067229 state 0x0 ref 1 00000100:00100000:1.0:1789760781.494063:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759907327645 time 759907067229 state 0x1 ref 1 00000100:00100000:1.0:1789760781.494092:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759907327645 time 759959381761 state 0x0 ref 1 00000100:00100000:1.0:1789760781.494104:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515801984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760781.494162:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25612288, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:1.0:1789760781.494172:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.494187:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760781.494191:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.494197:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.494210:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760781.534516:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25612288 -> 25616384 00010000:00010000:0.0:1789760781.534536:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.534549:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.534559:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.534566:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760781.534571:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.534579:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.534598:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.535571:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973570 is committed 00010000:00010000:0.0:1789760781.535998:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fd65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.536097:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515801984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41990us (42180us total) trans 4294973570 rc 0/0 00000100:00100000:0.0:1789760781.536108:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6282 00000100:00100000:0.0:1789760781.536118:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759907327645 time 759959381761 state 0x2 ref 0 00000100:00100000:2.0:1789760781.544352:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760781.544519:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802112 00000100:00100000:1.0:1789760781.544528:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802112 00000100:00100000:1.0:1789760781.544576:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759907327645 time 759959381761 state 0x0 ref 1 00000100:00100000:1.0:1789760781.544590:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759959642177 time 759959381761 state 0x1 ref 1 00000100:00100000:1.0:1789760781.544657:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759959642177 time 760009945133 state 0x0 ref 1 00000100:00100000:1.0:1789760781.544670:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760781.544755:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25616384, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:1.0:1789760781.544764:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.544779:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760781.544783:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.544789:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.544804:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.573106:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973571 is committed 00002000:00010000:2.0:1789760781.573185:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25616384 -> 25620480 00010000:00010000:2.0:1789760781.573200:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.573213:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.573223:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.573236:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760781.573241:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.573250:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.573271:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.573279:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.573372:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28698us (29022us total) trans 4294973571 rc 0/0 00000100:00100000:2.0:1789760781.573382:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6283 00000100:00100000:2.0:1789760781.573392:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759959642177 time 760009945133 state 0x2 ref 0 00000100:00100000:2.0:1789760781.581830:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.581903:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802240 00000100:00100000:3.0:1789760781.581914:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802240 00000100:00100000:3.0:1789760781.581979:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 759959642177 time 760009945133 state 0x0 ref 1 00000100:00100000:3.0:1789760781.581994:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760010205549 time 760009945133 state 0x1 ref 1 00000100:00100000:3.0:1789760781.582035:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760010205549 time 760047315734 state 0x0 ref 1 00000100:00100000:3.0:1789760781.582468:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515802240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.582526:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25620480, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.582536:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.582550:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.582554:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.582560:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.582578:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.629282:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973572 is committed 00002000:00010000:3.0:1789760781.629367:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25620480 -> 25624576 00010000:00010000:3.0:1789760781.629381:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.629394:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.629404:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.629411:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.629416:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.629425:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.629445:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.629453:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.629548:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515802240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47081us (47723us total) trans 4294973572 rc 0/0 00000100:00100000:3.0:1789760781.629558:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6284 00000100:00100000:3.0:1789760781.629568:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760010205549 time 760047315734 state 0x2 ref 0 00010000:00010000:3.0:1789760781.630459:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760781.641703:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.642304:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802368 00000100:00100000:3.0:1789760781.642315:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802368 00000100:00100000:3.0:1789760781.642371:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760010205549 time 760047315734 state 0x0 ref 1 00000100:00100000:3.0:1789760781.642388:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760047576150 time 760047315734 state 0x1 ref 1 00000100:00100000:3.0:1789760781.642463:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760047576150 time 760107751174 state 0x0 ref 1 00000100:00100000:3.0:1789760781.642476:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.642525:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25624576, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.642534:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.642551:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.642555:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.642562:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.642575:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.669152:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973573 is committed 00002000:00010000:3.0:1789760781.669363:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25624576 -> 25628672 00010000:00010000:3.0:1789760781.669379:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.669393:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.669403:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.669409:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.669414:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.669423:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.669443:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.669451:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.669583:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27104us (27884us total) trans 4294973573 rc 0/0 00000100:00100000:3.0:1789760781.669598:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6285 00000100:00100000:3.0:1789760781.669617:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760047576150 time 760107751174 state 0x2 ref 0 00000100:00100000:2.0:1789760781.676191:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.676280:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802496 00000100:00100000:3.0:1789760781.676290:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802496 00000100:00100000:3.0:1789760781.676355:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760047576150 time 760107751174 state 0x0 ref 1 00000100:00100000:3.0:1789760781.676371:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760108011590 time 760107751174 state 0x1 ref 1 00000100:00100000:3.0:1789760781.676400:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760108011590 time 760141689356 state 0x0 ref 1 00000100:00100000:3.0:1789760781.676413:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.676460:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25628672, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.676470:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.676485:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.676489:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.676495:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.676509:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.704723:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973574 is committed 00002000:00010000:3.0:1789760781.704849:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25628672 -> 25632768 00010000:00010000:3.0:1789760781.704866:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.704881:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.704889:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.704896:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.704900:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.704908:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.704928:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.704936:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.705024:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28608us (28839us total) trans 4294973574 rc 0/0 00000100:00100000:3.0:1789760781.705035:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6286 00000100:00100000:3.0:1789760781.705045:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760108011590 time 760141689356 state 0x2 ref 0 00000100:00100000:2.0:1789760781.708669:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760781.708866:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802624 00000100:00100000:0.0:1789760781.708875:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802624 00000100:00100000:0.0:1789760781.708927:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760108011590 time 760141689356 state 0x0 ref 1 00000100:00100000:0.0:1789760781.708942:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760141949772 time 760141689356 state 0x1 ref 1 00000100:00100000:0.0:1789760781.708972:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760141949772 time 760174261447 state 0x0 ref 1 00000100:00100000:0.0:1789760781.708984:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515802624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760781.709031:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25632768, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:0.0:1789760781.709040:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.709054:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.709058:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.709065:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.709078:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.743250:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25632768 -> 25636864 00010000:00010000:2.0:1789760781.743270:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743283:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743293:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743302:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760781.743307:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743317:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743338:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743347:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1fd88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.743444:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515802624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34457us (34781us total) trans 4294973575 rc 0/0 00000100:00100000:2.0:1789760781.743455:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6287 00000100:00100000:2.0:1789760781.743464:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760141949772 time 760174261447 state 0x2 ref 0 00000001:00080000:2.0:1789760781.743506:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973575 is committed 00000100:00100000:2.0:1789760781.749889:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760781.750234:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802752 00000100:00100000:0.0:1789760781.750493:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802752 00000100:00100000:0.0:1789760781.750545:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760141949772 time 760174261447 state 0x0 ref 1 00000100:00100000:0.0:1789760781.750562:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760174521863 time 760174261447 state 0x1 ref 1 00000100:00100000:0.0:1789760781.750618:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760174521863 time 760215905215 state 0x0 ref 1 00000100:00100000:0.0:1789760781.750632:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515802752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760781.750676:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25636864, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:0.0:1789760781.750695:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.750738:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.750743:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.750749:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.750763:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.794408:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25636864 -> 25640960 00010000:00010000:2.0:1789760781.794428:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.794442:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.794452:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.794461:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760781.794466:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.794476:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.794497:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.794505:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fd8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.794666:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515802752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43965us (44715us total) trans 4294973576 rc 0/0 00000100:00100000:2.0:1789760781.794689:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6288 00000100:00100000:2.0:1789760781.794700:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760174521863 time 760215905215 state 0x2 ref 0 00000001:00080000:3.0:1789760781.795235:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973576 is committed 00000100:00100000:2.0:1789760781.797874:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760781.798005:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515802880 00000100:00100000:1.0:1789760781.798016:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515802880 00000100:00100000:1.0:1789760781.798067:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760174521863 time 760215905215 state 0x0 ref 1 00000100:00100000:1.0:1789760781.798081:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760216165631 time 760215905215 state 0x1 ref 1 00000100:00100000:1.0:1789760781.798111:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760216165631 time 760263400284 state 0x0 ref 1 00000100:00100000:1.0:1789760781.798124:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515802880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760781.798169:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25640960, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:1.0:1789760781.798178:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.798192:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760781.798197:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.798203:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.798217:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.819594:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25640960 -> 25645056 00010000:00010000:1.0:1789760781.819624:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819637:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819651:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819659:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760781.819664:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819674:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819703:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.819740:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fd96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.819820:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515802880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21693us (21950us total) trans 4294973577 rc 0/0 00000100:00100000:1.0:1789760781.819830:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6289 00000100:00100000:1.0:1789760781.819839:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760216165631 time 760263400284 state 0x2 ref 0 00000001:00080000:3.0:1789760781.820031:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973577 is committed 00000100:00100000:1.0:1789760781.824477:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.824564:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803008 00000100:00100000:3.0:1789760781.824576:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803008 00000100:00100000:3.0:1789760781.824699:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760216165631 time 760263400284 state 0x0 ref 1 00000100:00100000:3.0:1789760781.824780:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760263660700 time 760263400284 state 0x1 ref 1 00000100:00100000:3.0:1789760781.824855:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760263660700 time 760290142725 state 0x0 ref 1 00000100:00100000:3.0:1789760781.824868:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515803008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.824917:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25645056, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc807 00010000:00010000:3.0:1789760781.824927:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.824941:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.824945:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.824952:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.824965:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760781.855218:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25645056 -> 25649152 00002000:00010000:0.0:1789760781.855227:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:0.0:1789760781.855230:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00010000:00010000:0.0:1789760781.855243:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.855255:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.855265:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.855273:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760781.855278:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.855287:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.855305:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.855314:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d1fd9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.855478:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515803008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30607us (31006us total) trans 4294973578 rc 0/0 00000100:00100000:0.0:1789760781.855489:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6290 00000100:00100000:0.0:1789760781.855499:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760263660700 time 760290142725 state 0x2 ref 0 00000001:00080000:1.0:1789760781.855575:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973578 is committed 00000100:00100000:0.0:1789760781.858793:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.858889:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803136 00000100:00100000:3.0:1789760781.858899:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803136 00000100:00100000:3.0:1789760781.858951:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760263660700 time 760290142725 state 0x0 ref 1 00000100:00100000:3.0:1789760781.858967:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760290403141 time 760290142725 state 0x1 ref 1 00000100:00100000:3.0:1789760781.859003:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760290403141 time 760324291730 state 0x0 ref 1 00000100:00100000:3.0:1789760781.859016:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.859064:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25649152, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760781.859074:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859090:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.859094:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.859101:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859114:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.887201:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25649152 -> 25653248 00010000:00010000:1.0:1789760781.887221:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887234:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887243:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887251:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760781.887257:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887267:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887288:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887298:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1fda4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.887467:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515803136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28448us (28677us total) trans 4294973579 rc 0/0 00000100:00100000:1.0:1789760781.887479:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6291 00000100:00100000:1.0:1789760781.887489:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760290403141 time 760324291730 state 0x2 ref 0 00000001:00080000:2.0:1789760781.890662:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973579 is committed 00000100:00100000:2.0:1789760781.891394:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760781.891518:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803264 00000100:00100000:3.0:1789760781.891527:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803264 00000100:00100000:3.0:1789760781.891583:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760290403141 time 760324291730 state 0x0 ref 1 00000100:00100000:3.0:1789760781.891611:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760324552146 time 760324291730 state 0x1 ref 1 00000100:00100000:3.0:1789760781.891676:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760324552146 time 760356963978 state 0x0 ref 1 00000100:00100000:3.0:1789760781.891700:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515803264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760781.891780:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25653248, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760781.891790:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.891805:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.891810:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.891817:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.891831:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.919413:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25653248 -> 25657344 00010000:00010000:3.0:1789760781.919438:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.919451:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.919460:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.919468:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.919473:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.919483:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.919508:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.919516:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.919735:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973580 is committed 00000100:00100000:3.0:1789760781.919825:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515803264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28123us (28431us total) trans 4294973580 rc 0/0 00000100:00100000:3.0:1789760781.919838:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6292 00000100:00100000:3.0:1789760781.919848:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760324552146 time 760356963978 state 0x2 ref 0 00000100:00100000:0.0:1789760781.923389:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760781.923493:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803392 00000100:00100000:2.0:1789760781.923504:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803392 00000100:00100000:2.0:1789760781.923564:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760324552146 time 760356963978 state 0x0 ref 1 00000100:00100000:2.0:1789760781.923580:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760357224394 time 760356963978 state 0x1 ref 1 00000100:00100000:2.0:1789760781.923654:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760357224394 time 760388941849 state 0x0 ref 1 00000100:00100000:2.0:1789760781.923668:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515803392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760781.923911:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25657344, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760781.923921:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.923937:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.923943:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.923950:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.923965:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.948521:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973581 is committed 00002000:00010000:0.0:1789760781.948701:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25657344 -> 25661440 00010000:00010000:0.0:1789760781.948778:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.948791:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.948801:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.948808:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760781.948813:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.948823:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.948843:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.948852:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0cde700/0x69dc3c3d61d1fdb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.948984:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515803392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25314us (25669us total) trans 4294973581 rc 0/0 00000100:00100000:0.0:1789760781.948997:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6293 00000100:00100000:0.0:1789760781.949007:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760357224394 time 760388941849 state 0x2 ref 0 00000100:00100000:0.0:1789760781.953434:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760781.953529:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803520 00000100:00100000:2.0:1789760781.953540:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803520 00000100:00100000:2.0:1789760781.953592:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760357224394 time 760388941849 state 0x0 ref 1 00000100:00100000:2.0:1789760781.953614:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760389202265 time 760388941849 state 0x1 ref 1 00000100:00100000:2.0:1789760781.953640:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760389202265 time 760418929191 state 0x0 ref 1 00000100:00100000:2.0:1789760781.953653:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515803520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760781.953749:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25661440, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760781.953758:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.953773:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.953777:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.953783:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.953797:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.994755:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973582 is committed 00002000:00010000:3.0:1789760781.994855:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25661440 -> 25665536 00010000:00010000:3.0:1789760781.994870:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.994883:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.994894:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.994902:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760781.994907:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.994921:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.994941:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.994951:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1fdb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.995043:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515803520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41387us (41614us total) trans 4294973582 rc 0/0 00000100:00100000:3.0:1789760781.995054:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6294 00000100:00100000:3.0:1789760781.995064:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760389202265 time 760418929191 state 0x2 ref 0 00000100:00100000:1.0:1789760782.000663:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.000940:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803648 00000100:00100000:2.0:1789760782.000950:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803648 00000100:00100000:3.0:1789760782.001127:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760389202265 time 760418929191 state 0x0 ref 1 00000100:00100000:3.0:1789760782.001143:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760419189607 time 760418929191 state 0x1 ref 1 00000100:00100000:3.0:1789760782.001216:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760419189607 time 760466503473 state 0x0 ref 1 00000100:00100000:3.0:1789760782.001230:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760782.001279:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25665536, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760782.001288:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.001309:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760782.001314:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.001320:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.001334:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760782.002411:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760782.093161:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973583 is committed 00002000:00010000:3.0:1789760782.098843:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25665536 -> 25669632 00010000:00010000:3.0:1789760782.098862:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.098874:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.098883:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.098890:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760782.098895:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.098906:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.098925:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.098933:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.099083:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 97850us (98423us total) trans 4294973583 rc 0/0 00000100:00100000:3.0:1789760782.099095:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6295 00000100:00100000:3.0:1789760782.099105:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760419189607 time 760466503473 state 0x2 ref 0 00000100:00100000:0.0:1789760782.116126:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760782.116218:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803776 00000100:00100000:3.0:1789760782.116229:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803776 00000100:00100000:3.0:1789760782.116285:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760419189607 time 760466503473 state 0x0 ref 1 00000100:00100000:3.0:1789760782.116302:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760466763889 time 760466503473 state 0x1 ref 1 00000100:00100000:3.0:1789760782.116335:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760466763889 time 760581623669 state 0x0 ref 1 00000100:00100000:3.0:1789760782.116348:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760782.116408:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25669632, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760782.116418:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.116437:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760782.116443:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.116450:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.116513:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.180308:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973584 is committed 00002000:00010000:3.0:1789760782.180750:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25669632 -> 25673728 00010000:00010000:3.0:1789760782.180766:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.180778:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.180786:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.180794:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760782.180798:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.180807:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.180826:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.180834:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.180966:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64614us (64843us total) trans 4294973584 rc 0/0 00000100:00100000:3.0:1789760782.180978:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6296 00000100:00100000:3.0:1789760782.180988:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760466763889 time 760581623669 state 0x2 ref 0 00000100:00100000:0.0:1789760782.184523:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760782.184608:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515803904 00000100:00100000:1.0:1789760782.184618:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515803904 00000100:00100000:1.0:1789760782.184670:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760466763889 time 760581623669 state 0x0 ref 1 00000100:00100000:1.0:1789760782.184696:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760581884085 time 760581623669 state 0x1 ref 1 00000100:00100000:1.0:1789760782.184752:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760581884085 time 760650040322 state 0x0 ref 1 00000100:00100000:1.0:1789760782.184765:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760782.184809:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25673728, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:1.0:1789760782.184818:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.184834:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760782.184838:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.184845:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.184859:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.215955:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973585 is committed 00002000:00010000:1.0:1789760782.216126:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25673728 -> 25677824 00010000:00010000:1.0:1789760782.216141:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.216153:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.216162:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.216169:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760782.216174:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.216184:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.216205:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.216214:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d1fdce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.216308:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515803904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31539us (31790us total) trans 4294973585 rc 0/0 00000100:00100000:1.0:1789760782.216753:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6297 00000100:00100000:1.0:1789760782.216764:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760581884085 time 760650040322 state 0x2 ref 0 00000100:00100000:1.0:1789760782.228026:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760782.228124:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804032 00000100:00100000:0.0:1789760782.228134:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804032 00000100:00100000:0.0:1789760782.228188:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760581884085 time 760650040322 state 0x0 ref 1 00000100:00100000:0.0:1789760782.228202:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760650300738 time 760650040322 state 0x1 ref 1 00000100:00100000:0.0:1789760782.228232:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760650300738 time 760693521444 state 0x0 ref 1 00000100:00100000:0.0:1789760782.228245:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515804032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760782.228292:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25677824, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:0.0:1789760782.228301:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.228317:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760782.228321:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.228328:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.228347:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.270968:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973586 is committed 00002000:00010000:1.0:1789760782.271063:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25677824 -> 25681920 00010000:00010000:1.0:1789760782.271079:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.271092:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.271101:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.271108:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760782.271112:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.271119:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.271138:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.271146:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fdd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.271418:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515804032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43170us (43396us total) trans 4294973586 rc 0/0 00000100:00100000:1.0:1789760782.271429:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6298 00000100:00100000:1.0:1789760782.271438:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760650300738 time 760693521444 state 0x2 ref 0 00000100:00100000:0.0:1789760782.275107:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.275319:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804160 00000100:00100000:2.0:1789760782.275329:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804160 00000100:00100000:2.0:1789760782.275386:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760650300738 time 760693521444 state 0x0 ref 1 00000100:00100000:2.0:1789760782.275402:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760693781860 time 760693521444 state 0x1 ref 1 00000100:00100000:2.0:1789760782.275475:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760693781860 time 760740762804 state 0x0 ref 1 00000100:00100000:2.0:1789760782.275487:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.275534:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25681920, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.275544:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.275560:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760782.275564:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.275569:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.275580:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.305800:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973587 is committed 00002000:00010000:0.0:1789760782.305895:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25681920 -> 25686016 00010000:00010000:0.0:1789760782.305912:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.305925:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.305934:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.305941:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.305947:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.305956:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.305975:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.305983:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d1fddc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.306100:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30610us (30997us total) trans 4294973587 rc 0/0 00000100:00100000:0.0:1789760782.306111:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6299 00000100:00100000:0.0:1789760782.306120:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760693781860 time 760740762804 state 0x2 ref 0 00000100:00100000:0.0:1789760782.310696:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760782.310868:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804288 00000100:00100000:3.0:1789760782.310878:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804288 00000100:00100000:2.0:1789760782.311018:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760693781860 time 760740762804 state 0x0 ref 1 00000100:00100000:2.0:1789760782.311034:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760741023220 time 760740762804 state 0x1 ref 1 00000100:00100000:2.0:1789760782.311065:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760741023220 time 760776353708 state 0x0 ref 1 00000100:00100000:2.0:1789760782.311078:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.311125:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25686016, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.311134:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.311152:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760782.311156:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.311663:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.311693:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.346428:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973588 is committed 00002000:00010000:1.0:1789760782.346765:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25686016 -> 25690112 00010000:00010000:1.0:1789760782.346782:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.346794:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.346803:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.346809:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760782.346814:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.346822:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.346843:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.346850:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1fde3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.346942:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35862us (36261us total) trans 4294973588 rc 0/0 00000100:00100000:1.0:1789760782.346952:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6300 00000100:00100000:1.0:1789760782.346962:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760741023220 time 760776353708 state 0x2 ref 0 00000100:00100000:1.0:1789760782.352438:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760782.352799:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804416 00000100:00100000:3.0:1789760782.352809:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804416 00000100:00100000:3.0:1789760782.352861:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760741023220 time 760776353708 state 0x0 ref 1 00000100:00100000:3.0:1789760782.352877:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760776614124 time 760776353708 state 0x1 ref 1 00000100:00100000:3.0:1789760782.352908:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760776614124 time 760818196892 state 0x0 ref 1 00000100:00100000:3.0:1789760782.352921:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760782.352974:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25690112, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760782.352982:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.352998:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760782.353002:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.353009:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.353029:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760782.380917:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25690112 -> 25694208 00000001:00080000:3.0:1789760782.380917:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973589 is committed 00010000:00010000:2.0:1789760782.380938:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.380952:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.380962:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.380970:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760782.380976:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.380985:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.381005:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.381013:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1fdea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.381111:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28187us (28678us total) trans 4294973589 rc 0/0 00000100:00100000:2.0:1789760782.381120:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6301 00000100:00100000:2.0:1789760782.381130:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760776614124 time 760818196892 state 0x2 ref 0 00000100:00100000:0.0:1789760782.385205:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.385303:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804544 00000100:00100000:2.0:1789760782.385318:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804544 00000100:00100000:2.0:1789760782.385440:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760776614124 time 760818196892 state 0x0 ref 1 00000100:00100000:2.0:1789760782.385459:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760818457308 time 760818196892 state 0x1 ref 1 00000100:00100000:2.0:1789760782.385527:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760818457308 time 760850815326 state 0x0 ref 1 00000100:00100000:2.0:1789760782.385540:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.385586:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25694208, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.385594:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.385610:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760782.385613:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.385620:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.385637:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.413349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973590 is committed 00002000:00010000:3.0:1789760782.413523:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25694208 -> 25698304 00010000:00010000:3.0:1789760782.413539:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.413552:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.413560:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.413568:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760782.413581:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.413592:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.413612:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.413619:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d1fdf1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.413756:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28213us (28557us total) trans 4294973590 rc 0/0 00000100:00100000:3.0:1789760782.413767:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6302 00000100:00100000:3.0:1789760782.413776:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760818457308 time 760850815326 state 0x2 ref 0 00000100:00100000:0.0:1789760782.417827:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760782.418011:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804672 00000100:00100000:1.0:1789760782.418022:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804672 00000100:00100000:1.0:1789760782.418077:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760818457308 time 760850815326 state 0x0 ref 1 00000100:00100000:1.0:1789760782.418093:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760851075742 time 760850815326 state 0x1 ref 1 00000100:00100000:1.0:1789760782.418177:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760851075742 time 760883464983 state 0x0 ref 1 00000100:00100000:1.0:1789760782.418200:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760782.418252:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25698304, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:1.0:1789760782.418261:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418285:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760782.418290:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.418296:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.418311:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.447494:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973591 is committed 00002000:00010000:3.0:1789760782.448603:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25698304 -> 25702400 00010000:00010000:3.0:1789760782.448620:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.448633:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.448641:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.448647:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760782.448652:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.448888:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.448912:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.448925:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1fdf8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.449059:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30867us (31239us total) trans 4294973591 rc 0/0 00000100:00100000:3.0:1789760782.449070:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6303 00000100:00100000:3.0:1789760782.449079:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760851075742 time 760883464983 state 0x2 ref 0 00000100:00100000:1.0:1789760782.453391:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760782.453584:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804800 00000100:00100000:0.0:1789760782.453595:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804800 00000100:00100000:0.0:1789760782.453649:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760851075742 time 760883464983 state 0x0 ref 1 00000100:00100000:0.0:1789760782.453664:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760883725399 time 760883464983 state 0x1 ref 1 00000100:00100000:0.0:1789760782.453702:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760883725399 time 760918991307 state 0x0 ref 1 00000100:00100000:1.0:1789760782.453976:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760782.454024:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25702400, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:1.0:1789760782.454033:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.454048:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760782.454052:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.454058:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.454071:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.484654:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25702400 -> 25706496 00010000:00010000:0.0:1789760782.484674:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.484698:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.484740:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.484755:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.484761:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.484770:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.484792:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.484800:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973592 is committed 00010000:00010000:0.0:1789760782.484801:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d1fdff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.484938:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515804800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31030us (31550us total) trans 4294973592 rc 0/0 00000100:00100000:1.0:1789760782.484949:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6304 00000100:00100000:1.0:1789760782.484959:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760883725399 time 760918991307 state 0x2 ref 0 00000100:00100000:0.0:1789760782.487944:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.488008:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515804928 00000100:00100000:2.0:1789760782.488018:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515804928 00000100:00100000:2.0:1789760782.488277:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760883725399 time 760918991307 state 0x0 ref 1 00000100:00100000:2.0:1789760782.488289:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760919251723 time 760918991307 state 0x1 ref 1 00000100:00100000:2.0:1789760782.488312:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760919251723 time 760953602644 state 0x0 ref 1 00000100:00100000:2.0:1789760782.488322:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515804928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.488364:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25706496, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.488373:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.488389:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760782.488393:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.488400:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.488413:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.513564:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973593 is committed 00002000:00010000:0.0:1789760782.513572:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25706496 -> 25710592 00010000:00010000:0.0:1789760782.513588:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.513601:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.513611:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.513618:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.513623:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.513632:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.513652:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.513661:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d1fe06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.513952:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515804928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25627us (26012us total) trans 4294973593 rc 0/0 00000100:00100000:0.0:1789760782.513963:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6305 00000100:00100000:0.0:1789760782.513973:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760919251723 time 760953602644 state 0x2 ref 0 00000100:00100000:0.0:1789760782.519843:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.519908:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805056 00000100:00100000:2.0:1789760782.519918:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805056 00000100:00100000:2.0:1789760782.519970:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760919251723 time 760953602644 state 0x0 ref 1 00000100:00100000:2.0:1789760782.519984:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760953863060 time 760953602644 state 0x1 ref 1 00000100:00100000:2.0:1789760782.520016:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760953863060 time 760985304930 state 0x0 ref 1 00000100:00100000:2.0:1789760782.520033:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515805056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.520079:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25710592, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.520092:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.520107:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760782.520112:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.520118:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.520131:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.549089:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973594 is committed 00002000:00010000:0.0:1789760782.549226:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25710592 -> 25714688 00010000:00010000:0.0:1789760782.549242:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.549254:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.549264:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.549276:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.549281:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.549290:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.549311:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.549319:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d1fe0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.549456:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515805056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29419us (29617us total) trans 4294973594 rc 0/0 00000100:00100000:0.0:1789760782.549466:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6306 00000100:00100000:0.0:1789760782.549476:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760953863060 time 760985304930 state 0x2 ref 0 00000100:00100000:1.0:1789760782.554663:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.555115:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805184 00000100:00100000:2.0:1789760782.555126:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805184 00000100:00100000:2.0:1789760782.555179:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760953863060 time 760985304930 state 0x0 ref 1 00000100:00100000:2.0:1789760782.555194:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760985565346 time 760985304930 state 0x1 ref 1 00000100:00100000:2.0:1789760782.555226:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760985565346 time 761020515167 state 0x0 ref 1 00000100:00100000:2.0:1789760782.555239:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515805184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.555286:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25714688, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.555295:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.555311:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.555316:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.555322:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.555336:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760782.579979:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25714688 -> 25718784 00010000:00010000:2.0:1789760782.580001:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.580015:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.580024:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.580037:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760782.580041:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.580050:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.580068:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.580076:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.580157:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515805184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24916us (25500us total) trans 4294973595 rc 0/0 00000100:00100000:2.0:1789760782.580168:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6307 00000100:00100000:2.0:1789760782.580178:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760985565346 time 761020515167 state 0x2 ref 0 00000001:00080000:2.0:1789760782.580571:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973595 is committed 00000100:00100000:1.0:1789760782.586358:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760782.586554:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805312 00000100:00100000:3.0:1789760782.586562:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805312 00000100:00100000:3.0:1789760782.586612:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 760985565346 time 761020515167 state 0x0 ref 1 00000100:00100000:3.0:1789760782.586777:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761020775583 time 761020515167 state 0x1 ref 1 00000100:00100000:3.0:1789760782.586795:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761020775583 time 761052085595 state 0x0 ref 1 00000100:00100000:3.0:1789760782.586804:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515805312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760782.586843:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25718784, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760782.586849:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.586862:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.586865:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.586870:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.586880:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.633693:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973596 is committed 00002000:00010000:3.0:1789760782.633848:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25718784 -> 25722880 00010000:00010000:3.0:1789760782.633863:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.633876:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.633884:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.633891:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760782.633895:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.633905:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.633924:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.633932:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.634031:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515805312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47223us (47676us total) trans 4294973596 rc 0/0 00000100:00100000:3.0:1789760782.634041:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6308 00000100:00100000:3.0:1789760782.634051:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761020775583 time 761052085595 state 0x2 ref 0 00000100:00100000:2.0:1789760782.640878:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760782.641188:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805440 00000100:00100000:3.0:1789760782.641198:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805440 00000100:00100000:3.0:1789760782.641248:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761020775583 time 761052085595 state 0x0 ref 1 00000100:00100000:3.0:1789760782.641262:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761052346011 time 761052085595 state 0x1 ref 1 00000100:00100000:3.0:1789760782.641324:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761052346011 time 761106612723 state 0x0 ref 1 00000100:00100000:3.0:1789760782.641337:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515805440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760782.641386:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25722880, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:3.0:1789760782.641395:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.641412:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.641417:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.641423:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.641437:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.653892:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760782.684825:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973597 is committed 00002000:00010000:1.0:1789760782.684942:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25722880 -> 25726976 00010000:00010000:1.0:1789760782.684958:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.684971:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.684979:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.684987:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760782.684993:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.685001:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.685020:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.685027:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1fe22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.685252:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515805440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43912us (44382us total) trans 4294973597 rc 0/0 00000100:00100000:1.0:1789760782.685263:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6309 00000100:00100000:1.0:1789760782.685273:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761052346011 time 761106612723 state 0x2 ref 0 00000100:00100000:1.0:1789760782.693932:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.694032:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805568 00000100:00100000:2.0:1789760782.694042:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805568 00000100:00100000:2.0:1789760782.694090:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761052346011 time 761106612723 state 0x0 ref 1 00000100:00100000:2.0:1789760782.694105:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761106873139 time 761106612723 state 0x1 ref 1 00000100:00100000:2.0:1789760782.694127:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761106873139 time 761159416997 state 0x0 ref 1 00000100:00100000:2.0:1789760782.694138:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515805568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.694186:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25726976, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.694196:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.694211:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.694215:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.694221:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.694235:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.726449:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973598 is committed 00002000:00010000:0.0:1789760782.726549:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25726976 -> 25731072 00010000:00010000:0.0:1789760782.726565:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.726577:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.726590:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.726610:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.726615:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.726624:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.726643:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.726650:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fe29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.726973:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515805568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32829us (33043us total) trans 4294973598 rc 0/0 00000100:00100000:0.0:1789760782.726983:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6310 00000100:00100000:0.0:1789760782.726992:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761106873139 time 761159416997 state 0x2 ref 0 00000100:00100000:0.0:1789760782.735417:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.735526:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805696 00000100:00100000:2.0:1789760782.735537:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805696 00000100:00100000:2.0:1789760782.735590:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761106873139 time 761159416997 state 0x0 ref 1 00000100:00100000:2.0:1789760782.735619:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761159677413 time 761159416997 state 0x1 ref 1 00000100:00100000:2.0:1789760782.735650:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761159677413 time 761200939387 state 0x0 ref 1 00000100:00100000:2.0:1789760782.735662:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f436d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515805696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.735969:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25731072, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.735979:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.735996:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.736000:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.736007:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.736021:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.760378:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25731072 -> 25735168 00010000:00010000:0.0:1789760782.760399:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.760419:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.760429:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.760437:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.760442:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.760451:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.760469:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.760477:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1fe30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.760568:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f436d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515805696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24902us (25155us total) trans 4294973599 rc 0/0 00000100:00100000:0.0:1789760782.760579:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6311 00000100:00100000:0.0:1789760782.760589:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761159677413 time 761200939387 state 0x2 ref 0 00000001:00080000:1.0:1789760782.761294:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973599 is committed 00000100:00100000:0.0:1789760782.764378:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.764684:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805824 00000100:00100000:2.0:1789760782.764695:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805824 00000100:00100000:2.0:1789760782.764917:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761159677413 time 761200939387 state 0x0 ref 1 00000100:00100000:2.0:1789760782.764932:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761201199803 time 761200939387 state 0x1 ref 1 00000100:00100000:2.0:1789760782.764961:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761201199803 time 761230250394 state 0x0 ref 1 00000100:00100000:2.0:1789760782.764973:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515805824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.765154:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25735168, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:2.0:1789760782.765206:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.765224:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.765230:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.765236:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.765249:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760782.834933:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25735168 -> 25739264 00010000:00010000:3.0:1789760782.834953:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.834966:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.834974:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.834981:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760782.834986:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.834995:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.835015:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.835023:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fe37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.835117:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92f434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515805824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70142us (70745us total) trans 4294973600 rc 0/0 00000100:00100000:3.0:1789760782.835202:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6312 00000100:00100000:3.0:1789760782.835214:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761201199803 time 761230250394 state 0x2 ref 0 00000001:00080000:1.0:1789760782.835760:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973600 is committed 00000100:00100000:3.0:1789760782.839878:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760782.839952:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515805952 00000100:00100000:0.0:1789760782.839963:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515805952 00000100:00100000:0.0:1789760782.840015:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761201199803 time 761230250394 state 0x0 ref 1 00000100:00100000:0.0:1789760782.840031:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761230510810 time 761230250394 state 0x1 ref 1 00000100:00100000:0.0:1789760782.840116:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761230510810 time 761305404810 state 0x0 ref 1 00000100:00100000:0.0:1789760782.840141:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515805952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760782.840192:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25739264, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc807 00010000:00010000:0.0:1789760782.840202:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.840220:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760782.840225:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.840232:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.840247:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.846238:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544420096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.846320:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.846367:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544420608:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.846396:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.846702:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760782.846743:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544420480:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.846770:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.846799:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544420224:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.846825:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.846843:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544420736:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.846864:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.846882:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760782.846888:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544420352:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.846908:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.846935:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420096 00000100:00100000:2.0:1789760782.846941:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420096 00000100:00100000:2.0:1789760782.846983:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760782.846992:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544420096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847049:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3323:x1876699544420096:12345-0@lo:400:kworker.0 Request processed in 58us (729us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.847057:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789760782.847090:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420608 00000100:00100000:2.0:1789760782.847094:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420608 00000100:00100000:2.0:1789760782.847125:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420224 00000100:00100000:2.0:1789760782.847230:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420224 00000100:00100000:2.0:1789760782.847254:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420352 00000100:00100000:2.0:1789760782.847257:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420352 00000100:00100000:2.0:1789760782.847274:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789760782.847281:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544420608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847315:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3323:x1876699544420608:12345-0@lo:400:kworker.0 Request processed in 34us (918us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.847321:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789760782.847334:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1789760782.847339:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544420224:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847368:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544420224:12345-0@lo:400:kworker.0 Request processed in 29us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.847373:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1789760782.847382:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:2.0:1789760782.847387:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544420352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847483:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c25b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1876699544420352:12345-0@lo:400:kworker.0 Request processed in 95us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.847489:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:2.0:1789760782.847511:0:8226:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420480 00000100:00100000:2.0:1789760782.847515:0:8226:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420480 00000100:00100000:2.0:1789760782.847542:0:8226:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420736 00000100:00100000:2.0:1789760782.847545:0:8226:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420736 00000100:00100000:2.0:1789760782.847567:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789760782.847574:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544420480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847689:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544420480:12345-0@lo:400:kworker.0 Request processed in 115us (917us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.847696:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789760782.847734:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789760782.847741:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c259f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544420736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847769:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c259f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544420736:12345-0@lo:400:kworker.0 Request processed in 29us (904us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.847775:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789760782.847812:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544420480:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847821:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:850d5753-05fb-4719-a6af-8cf215b926a5:3323:1876699544420096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847831:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3323:1876699544420608:0@lo:400:kworker.0 00000100:00080000:2.0:1789760782.847857:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760782.847861:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760782.847863:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760782.847889:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3320:1876699544420352:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847898:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544420224:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.847907:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544420736:0@lo:400:kworker.0 00000001:00080000:0.0:1789760782.876247:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973601 is committed 00002000:00010000:2.0:1789760782.876418:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25739264 -> 25743360 00002000:00010000:2.0:1789760782.876423:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:2.0:1789760782.876425:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00010000:00010000:2.0:1789760782.876438:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.876451:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.876466:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.876474:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760782.876480:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.876489:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.876508:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.876516:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1fe3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.876616:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515805952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36484us (36743us total) trans 4294973601 rc 0/0 00000100:00100000:2.0:1789760782.876627:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6313 00000100:00100000:2.0:1789760782.876636:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761230510810 time 761305404810 state 0x2 ref 0 00000100:00100000:0.0:1789760782.880051:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760782.880138:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806080 00000100:00100000:3.0:1789760782.880148:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806080 00000100:00100000:3.0:1789760782.880200:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761230510810 time 761305404810 state 0x0 ref 1 00000100:00100000:3.0:1789760782.880215:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761305665226 time 761305404810 state 0x1 ref 1 00000100:00100000:3.0:1789760782.880243:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761305665226 time 761345532658 state 0x0 ref 1 00000100:00100000:3.0:1789760782.880255:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515806080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760782.880303:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25743360, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:3.0:1789760782.880313:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.880337:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.880342:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.880348:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.880362:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.915433:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973602 is committed 00002000:00010000:2.0:1789760782.915625:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25743360 -> 25747456 00010000:00010000:2.0:1789760782.915639:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.915657:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.915666:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.915684:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760782.915688:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.915697:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.915733:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.915740:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1fe45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.915829:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515806080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35570us (35783us total) trans 4294973602 rc 0/0 00000100:00100000:2.0:1789760782.915840:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6314 00000100:00100000:2.0:1789760782.915850:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761305665226 time 761345532658 state 0x2 ref 0 00000100:00100000:0.0:1789760782.921818:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760782.922193:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806208 00000100:00100000:1.0:1789760782.922203:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806208 00000100:00100000:1.0:1789760782.922254:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761305665226 time 761345532658 state 0x0 ref 1 00000100:00100000:1.0:1789760782.922273:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761345793074 time 761345532658 state 0x1 ref 1 00000100:00100000:1.0:1789760782.922351:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761345793074 time 761387639450 state 0x0 ref 1 00000100:00100000:1.0:1789760782.922365:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515806208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760782.922410:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25747456, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760782.922420:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.922436:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.922440:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.922446:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.922458:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.962431:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973603 is committed 00002000:00010000:0.0:1789760782.962573:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25747456 -> 25751552 00010000:00010000:0.0:1789760782.962589:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.962613:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.962622:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.962630:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760782.962634:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.962644:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.962664:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.962683:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d1fe4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.963013:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515806208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40645us (41199us total) trans 4294973603 rc 0/0 00000100:00100000:0.0:1789760782.963024:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6315 00000100:00100000:0.0:1789760782.963033:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761345793074 time 761387639450 state 0x2 ref 0 00000100:00100000:3.0:1789760782.970441:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760782.970535:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806336 00000100:00100000:2.0:1789760782.970547:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806336 00000100:00100000:2.0:1789760782.970654:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761345793074 time 761387639450 state 0x0 ref 1 00000100:00100000:2.0:1789760782.970680:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761387899866 time 761387639450 state 0x1 ref 1 00000100:00100000:2.0:1789760782.970778:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761387899866 time 761436066943 state 0x0 ref 1 00000100:00100000:2.0:1789760782.970792:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515806336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760782.970839:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25751552, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760782.970847:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.970862:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.970867:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.970874:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.970888:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.000342:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973604 is committed 00002000:00010000:2.0:1789760783.000521:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25751552 -> 25755648 00010000:00010000:2.0:1789760783.000536:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.000550:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.000566:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.000574:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.000578:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.000588:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.000686:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.000695:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.000804:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515806336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30010us (30367us total) trans 4294973604 rc 0/0 00000100:00100000:2.0:1789760783.000814:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6316 00000100:00100000:2.0:1789760783.000824:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761387899866 time 761436066943 state 0x2 ref 0 00000100:00100000:1.0:1789760783.004306:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.004610:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806464 00000100:00100000:0.0:1789760783.004621:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806464 00000100:00100000:0.0:1789760783.004687:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761387899866 time 761436066943 state 0x0 ref 1 00000100:00100000:0.0:1789760783.004703:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761436327359 time 761436066943 state 0x1 ref 1 00000100:00100000:0.0:1789760783.004759:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761436327359 time 761470048091 state 0x0 ref 1 00000100:00100000:0.0:1789760783.004772:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805527480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515806464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.004818:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25755648, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.004828:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.004844:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760783.004848:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.004854:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.004869:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.031739:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973605 is committed 00002000:00010000:1.0:1789760783.031844:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25755648 -> 25759744 00010000:00010000:1.0:1789760783.031862:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.031874:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.031886:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.031893:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.031899:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.031908:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.031928:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.031935:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1fe5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.032192:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805527480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515806464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27416us (27889us total) trans 4294973605 rc 0/0 00000100:00100000:1.0:1789760783.032203:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6317 00000100:00100000:1.0:1789760783.032214:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761436327359 time 761470048091 state 0x2 ref 0 00000100:00100000:0.0:1789760783.035942:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760783.036200:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806592 00000100:00100000:3.0:1789760783.036211:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806592 00000100:00100000:3.0:1789760783.036264:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761436327359 time 761470048091 state 0x0 ref 1 00000100:00100000:3.0:1789760783.036278:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761470308507 time 761470048091 state 0x1 ref 1 00000100:00100000:3.0:1789760783.036337:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761470308507 time 761501624537 state 0x0 ref 1 00000100:00100000:3.0:1789760783.036350:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515806592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760783.036399:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25759744, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:3.0:1789760783.036409:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.036424:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760783.036428:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.036433:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.036447:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.068868:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973606 is committed 00002000:00010000:3.0:1789760783.069097:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25759744 -> 25763840 00010000:00010000:3.0:1789760783.069112:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.069125:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.069134:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.069142:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760783.069147:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.069156:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.069175:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.069184:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.069308:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515806592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32955us (33369us total) trans 4294973606 rc 0/0 00000100:00100000:3.0:1789760783.069319:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6318 00000100:00100000:3.0:1789760783.069330:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761470308507 time 761501624537 state 0x2 ref 0 00000100:00100000:3.0:1789760783.078838:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.078890:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806720 00000100:00100000:0.0:1789760783.078901:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806720 00000100:00100000:0.0:1789760783.078955:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761470308507 time 761501624537 state 0x0 ref 1 00000100:00100000:0.0:1789760783.078970:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761501884953 time 761501624537 state 0x1 ref 1 00000100:00100000:0.0:1789760783.079006:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761501884953 time 761544295450 state 0x0 ref 1 00000100:00100000:0.0:1789760783.079085:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515806720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.079277:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25763840, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.079287:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.079303:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760783.079308:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.079314:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.079328:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.107586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973607 is committed 00002000:00010000:1.0:1789760783.107686:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25763840 -> 25767936 00010000:00010000:1.0:1789760783.107703:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107934:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107944:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107952:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.107958:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107967:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107987:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.107994:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fe68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.108087:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515806720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29065us (29253us total) trans 4294973607 rc 0/0 00000100:00100000:1.0:1789760783.108098:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6319 00000100:00100000:1.0:1789760783.108108:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761501884953 time 761544295450 state 0x2 ref 0 00000100:00100000:1.0:1789760783.116076:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.116172:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806848 00000100:00100000:2.0:1789760783.116182:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806848 00000100:00100000:2.0:1789760783.116284:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761501884953 time 761544295450 state 0x0 ref 1 00000100:00100000:2.0:1789760783.116787:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761544555866 time 761544295450 state 0x1 ref 1 00000100:00100000:2.0:1789760783.116824:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761544555866 time 761582112033 state 0x0 ref 1 00000100:00100000:2.0:1789760783.116839:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515806848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.116897:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25767936, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760783.116906:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.116921:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760783.116926:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.116932:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.116946:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.151477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973608 is committed 00002000:00010000:0.0:1789760783.151539:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25767936 -> 25772032 00010000:00010000:0.0:1789760783.151555:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.151573:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.151583:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.151600:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.151606:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.151616:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.151635:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.151643:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d1fe6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.151771:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515806848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34931us (35700us total) trans 4294973608 rc 0/0 00000100:00100000:0.0:1789760783.151782:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6320 00000100:00100000:0.0:1789760783.151791:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761544555866 time 761582112033 state 0x2 ref 0 00000100:00100000:0.0:1789760783.156130:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.156686:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515806976 00000100:00100000:1.0:1789760783.156697:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515806976 00000100:00100000:1.0:1789760783.156876:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761544555866 time 761582112033 state 0x0 ref 1 00000100:00100000:1.0:1789760783.156890:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761582372449 time 761582112033 state 0x1 ref 1 00000100:00100000:1.0:1789760783.156919:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761582372449 time 761622208618 state 0x0 ref 1 00000100:00100000:1.0:1789760783.156932:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515806976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760783.156981:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25772032, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760783.156991:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.157010:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760783.157015:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.157021:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.157034:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.191457:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973609 is committed 00002000:00010000:0.0:1789760783.191700:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25772032 -> 25776128 00010000:00010000:0.0:1789760783.191742:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.191754:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.191760:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.191769:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.191773:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.191779:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.191793:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.191798:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d1fe76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.191963:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515806976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35029us (35840us total) trans 4294973609 rc 0/0 00000100:00100000:0.0:1789760783.191973:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6321 00000100:00100000:0.0:1789760783.191989:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761582372449 time 761622208618 state 0x2 ref 0 00000100:00100000:3.0:1789760783.195508:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.195661:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807104 00000100:00100000:2.0:1789760783.195683:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807104 00000100:00100000:2.0:1789760783.195768:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761582372449 time 761622208618 state 0x0 ref 1 00000100:00100000:2.0:1789760783.195783:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761622469034 time 761622208618 state 0x1 ref 1 00000100:00100000:2.0:1789760783.195805:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761622469034 time 761661094967 state 0x0 ref 1 00000100:00100000:2.0:1789760783.195817:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515807104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.195868:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25776128, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760783.195878:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.195893:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760783.195898:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.195905:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.195920:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.224531:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25776128 -> 25780224 00000001:00080000:1.0:1789760783.224544:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973610 is committed 00010000:00010000:3.0:1789760783.224551:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.224563:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.224572:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.224580:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760783.224585:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.224594:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.224612:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.224620:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1fe7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.224792:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515807104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28971us (29288us total) trans 4294973610 rc 0/0 00000100:00100000:3.0:1789760783.224803:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6322 00000100:00100000:3.0:1789760783.224812:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761622469034 time 761661094967 state 0x2 ref 0 00000100:00100000:1.0:1789760783.228264:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.228479:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807232 00000100:00100000:0.0:1789760783.228489:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807232 00000100:00100000:0.0:1789760783.228543:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761622469034 time 761661094967 state 0x0 ref 1 00000100:00100000:0.0:1789760783.228558:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761661355383 time 761661094967 state 0x1 ref 1 00000100:00100000:0.0:1789760783.228596:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761661355383 time 761693874347 state 0x0 ref 1 00000100:00100000:0.0:1789760783.228613:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515807232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.228663:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25780224, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.228681:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.228698:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760783.228702:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.228851:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.228868:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.256905:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973611 is committed 00002000:00010000:0.0:1789760783.257474:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25780224 -> 25784320 00010000:00010000:0.0:1789760783.257490:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257502:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257510:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257518:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.257523:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257531:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257550:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.257557:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fe84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.257785:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515807232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29168us (29524us total) trans 4294973611 rc 0/0 00000100:00100000:0.0:1789760783.257796:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6323 00000100:00100000:0.0:1789760783.257805:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761661355383 time 761693874347 state 0x2 ref 0 00000100:00100000:0.0:1789760783.261574:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760783.262313:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807360 00000100:00100000:3.0:1789760783.262324:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807360 00000100:00100000:3.0:1789760783.262376:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761661355383 time 761693874347 state 0x0 ref 1 00000100:00100000:3.0:1789760783.262392:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761694134763 time 761693874347 state 0x1 ref 1 00000100:00100000:3.0:1789760783.262450:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761694134763 time 761727738882 state 0x0 ref 1 00000100:00100000:3.0:1789760783.262464:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515807360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760783.262513:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25784320, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:3.0:1789760783.262522:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.262538:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760783.262542:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.262548:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.262561:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.281196:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.281371:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807616 00000100:00100000:2.0:1789760783.281382:0:5855:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807616 00000100:00100000:2.0:1789760783.281503:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 496 00000100:00100000:2.0:1789760783.281514:0:5855:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515807616:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760783.281597:0:5855:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515807616:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 85us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760783.281606:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 496 00000100:00080000:3.0:1789760783.282565:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760783.285518:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.285610:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807488 00000100:00100000:1.0:1789760783.285621:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807488 00000100:00100000:1.0:1789760783.285691:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 408 00000100:00100000:1.0:1789760783.285702:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515807488:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760783.285793:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2365:x1876699515807488:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 93us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760783.285801:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 408 00000100:00100000:2.0:1789760783.287340:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.287465:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807744 00000100:00100000:1.0:1789760783.287474:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807744 00000100:00100000:1.0:1789760783.287546:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 497 00000100:00100000:1.0:1789760783.287561:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2365:x1876699515807744:12345-192.168.203.46@tcp:400:kworker.0 00000100:00080000:3.0:1789760783.287565:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760783.287635:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c258000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2365:x1876699515807744:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 78us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760783.287642:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 497 00002000:00010000:2.0:1789760783.298985:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25784320 -> 25788416 00010000:00010000:2.0:1789760783.299006:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.299019:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.299028:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.299037:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.299042:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.299051:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.299071:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.299079:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fe8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.299172:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515807360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36706us (37602us total) trans 4294973612 rc 0/0 00000100:00100000:2.0:1789760783.299182:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6324 00000100:00100000:2.0:1789760783.299192:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761694134763 time 761727738882 state 0x2 ref 0 00000001:00080000:1.0:1789760783.299552:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973612 is committed 00000100:00100000:2.0:1789760783.302429:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.302604:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515807872 00000100:00100000:1.0:1789760783.302614:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515807872 00000100:00100000:1.0:1789760783.302689:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761694134763 time 761727738882 state 0x0 ref 1 00000100:00100000:1.0:1789760783.302704:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761727999298 time 761727738882 state 0x1 ref 1 00000100:00100000:1.0:1789760783.302775:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761727999298 time 761768063741 state 0x0 ref 1 00000100:00100000:1.0:1789760783.302788:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515807872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760783.302836:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25788416, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760783.302845:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.302867:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760783.302871:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.302878:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.302892:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.339181:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973613 is committed 00002000:00010000:0.0:1789760783.339267:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25788416 -> 25792512 00010000:00010000:0.0:1789760783.339283:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.339301:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.339310:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.339318:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.339323:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.339332:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.339352:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.339360:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1fe92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.339512:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515807872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36715us (37081us total) trans 4294973613 rc 0/0 00000100:00100000:0.0:1789760783.339523:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6325 00000100:00100000:0.0:1789760783.339533:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761727999298 time 761768063741 state 0x2 ref 0 00000100:00100000:0.0:1789760783.346578:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760783.346876:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808000 00000100:00100000:3.0:1789760783.346886:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808000 00000100:00100000:2.0:1789760783.348060:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761727999298 time 761768063741 state 0x0 ref 1 00000100:00100000:2.0:1789760783.348078:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761768324157 time 761768063741 state 0x1 ref 1 00000100:00100000:2.0:1789760783.348118:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761768324157 time 761813407083 state 0x0 ref 1 00000100:00100000:2.0:1789760783.348132:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515808000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.348183:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25792512, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760783.348192:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.348208:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760783.348212:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.348218:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.348231:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760783.389264:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25792512 -> 25796608 00010000:00010000:2.0:1789760783.389283:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.389295:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.389373:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.389381:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.389386:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.389396:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.389416:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.389424:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d1fe99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.389514:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515808000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41378us (43007us total) trans 4294973614 rc 0/0 00000100:00100000:2.0:1789760783.389525:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6326 00000100:00100000:2.0:1789760783.389535:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761768324157 time 761813407083 state 0x2 ref 0 00000001:00080000:2.0:1789760783.389573:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973614 is committed 00000100:00100000:2.0:1789760783.397008:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.399505:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808128 00000100:00100000:0.0:1789760783.399520:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808128 00000100:00100000:0.0:1789760783.399588:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761768324157 time 761813407083 state 0x0 ref 1 00000100:00100000:0.0:1789760783.399604:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761813667499 time 761813407083 state 0x1 ref 1 00000100:00100000:0.0:1789760783.399698:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761813667499 time 761864986376 state 0x0 ref 1 00000100:00100000:0.0:1789760783.399733:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515808128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.399779:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25796608, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.399788:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.399804:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760783.399808:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.399814:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.399826:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.438941:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973615 is committed 00002000:00010000:0.0:1789760783.439069:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25796608 -> 25800704 00010000:00010000:0.0:1789760783.439086:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.439099:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.439108:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.439115:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.439119:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.439129:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.439149:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.439157:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d1fea0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.439284:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515808128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39571us (42282us total) trans 4294973615 rc 0/0 00000100:00100000:0.0:1789760783.439296:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6327 00000100:00100000:0.0:1789760783.439429:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761813667499 time 761864986376 state 0x2 ref 0 00000100:00100000:2.0:1789760783.443254:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760783.443438:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808256 00000100:00100000:3.0:1789760783.443448:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808256 00000100:00100000:3.0:1789760783.443502:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761813667499 time 761864986376 state 0x0 ref 1 00000100:00100000:3.0:1789760783.443517:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761865246792 time 761864986376 state 0x1 ref 1 00000100:00100000:3.0:1789760783.443579:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761865246792 time 761908867300 state 0x0 ref 1 00000100:00100000:3.0:1789760783.443601:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760783.443648:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25800704, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:3.0:1789760783.443657:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.443681:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760783.443686:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.443692:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.443725:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.473899:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25800704 -> 25804800 00010000:00010000:3.0:1789760783.473918:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.473930:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.473939:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.473946:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760783.473951:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.473960:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.473978:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.473986:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1fea7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.474081:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515808256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30486us (30830us total) trans 4294973616 rc 0/0 00000100:00100000:3.0:1789760783.474092:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6328 00000100:00100000:3.0:1789760783.474103:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761865246792 time 761908867300 state 0x2 ref 0 00000001:00080000:0.0:1789760783.474182:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973616 is committed 00000100:00100000:0.0:1789760783.478259:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.478764:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808384 00000100:00100000:1.0:1789760783.478773:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808384 00000100:00100000:1.0:1789760783.478830:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761865246792 time 761908867300 state 0x0 ref 1 00000100:00100000:1.0:1789760783.478846:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761909127716 time 761908867300 state 0x1 ref 1 00000100:00100000:1.0:1789760783.478876:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761909127716 time 761944164801 state 0x0 ref 1 00000100:00100000:1.0:1789760783.478887:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760783.478935:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25804800, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760783.478943:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.478959:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760783.478964:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.478970:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.478985:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.510818:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973617 is committed 00002000:00010000:1.0:1789760783.511223:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25804800 -> 25808896 00010000:00010000:1.0:1789760783.511241:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.511254:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.511263:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.511271:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.511276:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.511286:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.511375:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.511384:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1feae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.511558:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32665us (33301us total) trans 4294973617 rc 0/0 00000100:00100000:1.0:1789760783.511569:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6329 00000100:00100000:1.0:1789760783.511578:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761909127716 time 761944164801 state 0x2 ref 0 00000100:00100000:1.0:1789760783.515167:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.515293:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808512 00000100:00100000:2.0:1789760783.515304:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808512 00000100:00100000:2.0:1789760783.515419:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761909127716 time 761944164801 state 0x0 ref 1 00000100:00100000:2.0:1789760783.515435:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761944425217 time 761944164801 state 0x1 ref 1 00000100:00100000:2.0:1789760783.515466:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761944425217 time 761980755417 state 0x0 ref 1 00000100:00100000:2.0:1789760783.515477:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.515526:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25808896, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760783.515536:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.515550:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760783.515555:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.515561:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.515574:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.548251:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973618 is committed 00002000:00010000:3.0:1789760783.548402:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25808896 -> 25812992 00010000:00010000:3.0:1789760783.548418:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.548432:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.548446:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.548454:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760783.548459:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.548468:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.548488:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.548497:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d1feb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.548692:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33210us (33529us total) trans 4294973618 rc 0/0 00000100:00100000:3.0:1789760783.548703:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6330 00000100:00100000:3.0:1789760783.549065:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761944425217 time 761980755417 state 0x2 ref 0 00000100:00100000:3.0:1789760783.553296:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.553384:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808640 00000100:00100000:0.0:1789760783.553394:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808640 00000100:00100000:0.0:1789760783.553449:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761944425217 time 761980755417 state 0x0 ref 1 00000100:00100000:0.0:1789760783.553464:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761981015833 time 761980755417 state 0x1 ref 1 00000100:00100000:0.0:1789760783.553491:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761981015833 time 762018780128 state 0x0 ref 1 00000100:00100000:0.0:1789760783.553503:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.553553:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25812992, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.553563:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.553579:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.553583:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.553589:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.553603:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.587267:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973619 is committed 00002000:00010000:2.0:1789760783.587505:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25812992 -> 25817088 00010000:00010000:2.0:1789760783.587520:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.587533:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.587543:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.587550:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.587556:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.587569:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.587656:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.587666:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d1febc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.587884:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34378us (34593us total) trans 4294973619 rc 0/0 00000100:00100000:2.0:1789760783.587894:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6331 00000100:00100000:2.0:1789760783.587905:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761981015833 time 762018780128 state 0x2 ref 0 00000100:00100000:0.0:1789760783.591644:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760783.591748:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808768 00000100:00100000:3.0:1789760783.591759:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808768 00000100:00100000:3.0:1789760783.591812:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 761981015833 time 762018780128 state 0x0 ref 1 00000100:00100000:3.0:1789760783.591833:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762019040544 time 762018780128 state 0x1 ref 1 00000100:00100000:3.0:1789760783.591858:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762019040544 time 762057147274 state 0x0 ref 1 00000100:00100000:3.0:1789760783.591871:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515808768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760783.591921:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25817088, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:3.0:1789760783.591930:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.591947:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.591952:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.591959:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.591983:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760783.623653:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25817088 -> 25821184 00010000:00010000:2.0:1789760783.623680:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.623694:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.623703:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.623731:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.623737:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.623746:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.623766:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.623774:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d1fec3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.623861:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515808768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31987us (32223us total) trans 4294973620 rc 0/0 00000100:00100000:2.0:1789760783.623872:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6332 00000100:00100000:2.0:1789760783.623882:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762019040544 time 762057147274 state 0x2 ref 0 00000001:00080000:0.0:1789760783.624056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973620 is committed 00000100:00100000:3.0:1789760783.628751:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.628807:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515808896 00000100:00100000:0.0:1789760783.628816:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515808896 00000100:00100000:0.0:1789760783.628868:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762019040544 time 762057147274 state 0x0 ref 1 00000100:00100000:0.0:1789760783.628883:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762057407690 time 762057147274 state 0x1 ref 1 00000100:00100000:0.0:1789760783.628913:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762057407690 time 762094202227 state 0x0 ref 1 00000100:00100000:0.0:1789760783.628925:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515808896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.629067:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25821184, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.629076:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.629093:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.629097:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.629103:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.629117:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.666059:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973621 is committed 00002000:00010000:1.0:1789760783.666220:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25821184 -> 25825280 00010000:00010000:1.0:1789760783.666242:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.666257:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.666270:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.666278:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.666283:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.666295:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.666317:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.666329:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1feca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.666425:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515808896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37498us (37722us total) trans 4294973621 rc 0/0 00000100:00100000:1.0:1789760783.666438:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6333 00000100:00100000:1.0:1789760783.666451:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762057407690 time 762094202227 state 0x2 ref 0 00000100:00100000:3.0:1789760783.671516:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.671613:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809024 00000100:00100000:2.0:1789760783.671623:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809024 00000100:00100000:2.0:1789760783.671684:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762057407690 time 762094202227 state 0x0 ref 1 00000100:00100000:2.0:1789760783.671699:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762094462643 time 762094202227 state 0x1 ref 1 00000100:00100000:2.0:1789760783.671959:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762094462643 time 762137246933 state 0x0 ref 1 00000100:00100000:2.0:1789760783.671973:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.672026:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25825280, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760783.672035:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.672050:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.672055:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.672061:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.672075:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.678890:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760783.691185:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973622 is committed 00002000:00010000:2.0:1789760783.691404:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25825280 -> 25829376 00010000:00010000:2.0:1789760783.691421:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.691434:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.691444:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.691451:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.691456:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.691465:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.691485:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.691493:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d1fed1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.691692:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 19717us (20180us total) trans 4294973622 rc 0/0 00000100:00100000:2.0:1789760783.691703:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6334 00000100:00100000:2.0:1789760783.691744:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762094462643 time 762137246933 state 0x2 ref 0 00000100:00100000:2.0:1789760783.695224:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.695479:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809152 00000100:00100000:1.0:1789760783.695488:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809152 00000100:00100000:1.0:1789760783.695539:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762094462643 time 762137246933 state 0x0 ref 1 00000100:00100000:1.0:1789760783.695554:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762137507349 time 762137246933 state 0x1 ref 1 00000100:00100000:1.0:1789760783.695583:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762137507349 time 762160872177 state 0x0 ref 1 00000100:00100000:1.0:1789760783.695603:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515809152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760783.695650:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25829376, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760783.695658:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.695683:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.695688:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.695695:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.695884:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.723535:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973623 is committed 00002000:00010000:1.0:1789760783.723775:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25829376 -> 25833472 00010000:00010000:1.0:1789760783.723791:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.723804:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.723813:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.723821:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.723826:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.723835:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.723856:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.723863:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fed8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.723995:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515809152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28390us (28775us total) trans 4294973623 rc 0/0 00000100:00100000:1.0:1789760783.724006:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6335 00000100:00100000:1.0:1789760783.724016:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762137507349 time 762160872177 state 0x2 ref 0 00000100:00100000:3.0:1789760783.727482:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.727925:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809280 00000100:00100000:0.0:1789760783.727936:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809280 00000100:00100000:0.0:1789760783.727994:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762137507349 time 762160872177 state 0x0 ref 1 00000100:00100000:0.0:1789760783.728010:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762161132593 time 762160872177 state 0x1 ref 1 00000100:00100000:0.0:1789760783.728073:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762161132593 time 762193361669 state 0x0 ref 1 00000100:00100000:0.0:1789760783.728085:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.728133:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25833472, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:0.0:1789760783.728142:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.728158:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.728162:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.728169:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.728188:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.753418:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973624 is committed 00002000:00010000:0.0:1789760783.753504:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25833472 -> 25837568 00010000:00010000:0.0:1789760783.753520:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753534:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753544:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753552:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.753557:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753566:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753652:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.753662:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d1fedf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.753832:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25745us (26357us total) trans 4294973624 rc 0/0 00000100:00100000:0.0:1789760783.753843:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6336 00000100:00100000:0.0:1789760783.753853:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762161132593 time 762193361669 state 0x2 ref 0 00000100:00100000:3.0:1789760783.757873:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.757974:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809408 00000100:00100000:1.0:1789760783.757984:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809408 00000100:00100000:1.0:1789760783.758034:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762161132593 time 762193361669 state 0x0 ref 1 00000100:00100000:1.0:1789760783.758049:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762193622085 time 762193361669 state 0x1 ref 1 00000100:00100000:1.0:1789760783.758117:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762193622085 time 762223405470 state 0x0 ref 1 00000100:00100000:1.0:1789760783.758130:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760783.758176:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25837568, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760783.758186:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.758202:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.758206:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.758211:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.758225:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.782093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973625 is committed 00002000:00010000:1.0:1789760783.782196:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25837568 -> 25841664 00010000:00010000:1.0:1789760783.782212:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.782224:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.782234:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.782241:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.782247:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.782256:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.782278:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.782286:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fee6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.782455:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24321us (24585us total) trans 4294973625 rc 0/0 00000100:00100000:1.0:1789760783.782469:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6337 00000100:00100000:1.0:1789760783.782477:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762193622085 time 762223405470 state 0x2 ref 0 00000100:00100000:3.0:1789760783.785782:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.785855:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809536 00000100:00100000:2.0:1789760783.785865:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809536 00000100:00100000:2.0:1789760783.785917:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762193622085 time 762223405470 state 0x0 ref 1 00000100:00100000:2.0:1789760783.785943:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762223665886 time 762223405470 state 0x1 ref 1 00000100:00100000:2.0:1789760783.785968:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762223665886 time 762251257065 state 0x0 ref 1 00000100:00100000:2.0:1789760783.785980:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.786027:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25841664, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:2.0:1789760783.786035:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.786050:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.786055:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.786061:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.786075:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.811473:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25841664 -> 25845760 00010000:00010000:3.0:1789760783.811495:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.811513:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.811522:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.811529:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760783.811535:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.811543:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.811561:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.811569:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d1feed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.811674:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515809536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25683us (25889us total) trans 4294973626 rc 0/0 00000100:00100000:3.0:1789760783.811687:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6338 00000100:00100000:3.0:1789760783.811696:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762223665886 time 762251257065 state 0x2 ref 0 00000001:00080000:3.0:1789760783.814000:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973626 is committed 00000100:00100000:0.0:1789760783.815218:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760783.815322:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809664 00000100:00100000:1.0:1789760783.815331:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809664 00000100:00100000:1.0:1789760783.815382:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762223665886 time 762251257065 state 0x0 ref 1 00000100:00100000:1.0:1789760783.815398:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762251517481 time 762251257065 state 0x1 ref 1 00000100:00100000:1.0:1789760783.815425:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762251517481 time 762280714402 state 0x0 ref 1 00000100:00100000:1.0:1789760783.815438:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760783.815484:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25845760, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc807 00010000:00010000:1.0:1789760783.815494:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.815510:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.815515:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.815521:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.815535:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.841224:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25845760 -> 25849856 00002000:00010000:1.0:1789760783.841233:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:1.0:1789760783.841236:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00010000:00010000:1.0:1789760783.841249:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.841262:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.841271:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.841279:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760783.841284:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.841293:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.841384:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.841394:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1fef4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.841485:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515809664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26045us (26268us total) trans 4294973627 rc 0/0 00000100:00100000:1.0:1789760783.841496:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6339 00000100:00100000:1.0:1789760783.841506:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762251517481 time 762280714402 state 0x2 ref 0 00000001:00080000:2.0:1789760783.843065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973627 is committed 00000100:00100000:3.0:1789760783.844825:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.844991:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809792 00000100:00100000:2.0:1789760783.845001:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809792 00000100:00100000:2.0:1789760783.845053:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762251517481 time 762280714402 state 0x0 ref 1 00000100:00100000:2.0:1789760783.845068:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762280974818 time 762280714402 state 0x1 ref 1 00000100:00100000:2.0:1789760783.845098:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762280974818 time 762310387561 state 0x0 ref 1 00000100:00100000:2.0:1789760783.845110:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.845158:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25849856, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760783.845166:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.845182:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.845187:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.845193:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.845208:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.875608:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973628 is committed 00002000:00010000:2.0:1789760783.875852:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25849856 -> 25853952 00010000:00010000:2.0:1789760783.875867:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.875881:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.875890:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.875898:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.875903:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.875912:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.875932:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.875939:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1fefb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.876036:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515809792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30923us (31215us total) trans 4294973628 rc 0/0 00000100:00100000:2.0:1789760783.876046:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6340 00000100:00100000:2.0:1789760783.876055:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762280974818 time 762310387561 state 0x2 ref 0 00000100:00100000:1.0:1789760783.880898:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.881000:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515809920 00000100:00100000:0.0:1789760783.881010:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515809920 00000100:00100000:0.0:1789760783.881065:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762280974818 time 762310387561 state 0x0 ref 1 00000100:00100000:0.0:1789760783.881080:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762310647977 time 762310387561 state 0x1 ref 1 00000100:00100000:0.0:1789760783.881140:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762310647977 time 762346428432 state 0x0 ref 1 00000100:00100000:0.0:1789760783.881153:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515809920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.881201:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25853952, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:0.0:1789760783.881210:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.881227:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.881232:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.881239:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.881253:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.907961:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973629 is committed 00002000:00010000:0.0:1789760783.908066:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25853952 -> 25858048 00010000:00010000:0.0:1789760783.908081:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.908094:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.908103:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.908110:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.908115:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.908125:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.908144:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.908151:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.908246:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515809920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27090us (27354us total) trans 4294973629 rc 0/0 00000100:00100000:0.0:1789760783.908260:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6341 00000100:00100000:0.0:1789760783.908270:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762310647977 time 762346428432 state 0x2 ref 0 00000100:00100000:1.0:1789760783.912035:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760783.912114:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810048 00000100:00100000:0.0:1789760783.912124:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810048 00000100:00100000:0.0:1789760783.912178:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762310647977 time 762346428432 state 0x0 ref 1 00000100:00100000:0.0:1789760783.912192:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762346688848 time 762346428432 state 0x1 ref 1 00000100:00100000:0.0:1789760783.912220:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762346688848 time 762377509424 state 0x0 ref 1 00000100:00100000:0.0:1789760783.912233:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515810048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760783.912282:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25858048, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:0.0:1789760783.912291:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.912306:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.912323:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.912330:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.912343:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.941293:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973630 is committed 00002000:00010000:0.0:1789760783.941466:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25858048 -> 25862144 00010000:00010000:0.0:1789760783.941482:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.941496:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.941505:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.941512:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760783.941517:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.941526:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.941545:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.941553:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.941797:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515810048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29560us (29766us total) trans 4294973630 rc 0/0 00000100:00100000:0.0:1789760783.941808:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6342 00000100:00100000:0.0:1789760783.941818:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762346688848 time 762377509424 state 0x2 ref 0 00000100:00100000:1.0:1789760783.946233:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760783.946311:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810176 00000100:00100000:2.0:1789760783.946319:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810176 00000100:00100000:2.0:1789760783.946361:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762346688848 time 762377509424 state 0x0 ref 1 00000100:00100000:2.0:1789760783.946375:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762377769840 time 762377509424 state 0x1 ref 1 00000100:00100000:2.0:1789760783.946433:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762377769840 time 762411721656 state 0x0 ref 1 00000100:00100000:2.0:1789760783.946453:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515810176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760783.946502:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25862144, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760783.946511:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.946526:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.946530:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.946536:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.946548:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.967852:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973631 is committed 00002000:00010000:2.0:1789760783.967915:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25862144 -> 25866240 00010000:00010000:2.0:1789760783.967931:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967944:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967954:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967961:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760783.967966:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967975:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.967993:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.968001:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.968088:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515810176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 21632us (21858us total) trans 4294973631 rc 0/0 00000100:00100000:2.0:1789760783.968098:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6343 00000100:00100000:2.0:1789760783.968108:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762377769840 time 762411721656 state 0x2 ref 0 00000100:00100000:1.0:1789760783.970923:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760783.971144:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810304 00000100:00100000:3.0:1789760783.971155:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810304 00000100:00100000:3.0:1789760783.971208:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762377769840 time 762411721656 state 0x0 ref 1 00000100:00100000:3.0:1789760783.971223:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762411982072 time 762411721656 state 0x1 ref 1 00000100:00100000:3.0:1789760783.971251:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762411982072 time 762436540254 state 0x0 ref 1 00000100:00100000:3.0:1789760783.971262:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515810304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760783.971318:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25866240, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:3.0:1789760783.971328:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.971344:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.971348:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.971355:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.971368:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.017514:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973632 is committed 00002000:00010000:3.0:1789760784.017842:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25866240 -> 25870336 00010000:00010000:3.0:1789760784.017858:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.017869:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.017878:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.017885:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760784.017890:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.017898:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.017918:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.017926:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.018062:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515810304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46796us (47143us total) trans 4294973632 rc 0/0 00000100:00100000:3.0:1789760784.018072:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6344 00000100:00100000:3.0:1789760784.018081:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762411982072 time 762436540254 state 0x2 ref 0 00000100:00100000:1.0:1789760784.022226:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.022394:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810432 00000100:00100000:2.0:1789760784.022403:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810432 00000100:00100000:2.0:1789760784.022471:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762411982072 time 762436540254 state 0x0 ref 1 00000100:00100000:2.0:1789760784.022486:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762436800670 time 762436540254 state 0x1 ref 1 00000100:00100000:2.0:1789760784.022564:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762436800670 time 762487852194 state 0x0 ref 1 00000100:00100000:2.0:1789760784.022577:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515810432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.022635:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25870336, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.022643:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.022658:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760784.022662:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.022679:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.022692:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760784.023364:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760784.057313:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973633 is committed 00002000:00010000:2.0:1789760784.057516:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25870336 -> 25874432 00010000:00010000:2.0:1789760784.057532:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.057547:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.057557:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.057565:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.057570:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.057579:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.057662:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.057679:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.057853:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515810432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35273us (35631us total) trans 4294973633 rc 0/0 00000100:00100000:2.0:1789760784.057864:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6345 00000100:00100000:2.0:1789760784.057873:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762436800670 time 762487852194 state 0x2 ref 0 00000100:00100000:0.0:1789760784.061135:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.061841:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810560 00000100:00100000:2.0:1789760784.061853:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810560 00000100:00100000:2.0:1789760784.061913:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762436800670 time 762487852194 state 0x0 ref 1 00000100:00100000:2.0:1789760784.061928:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762488112610 time 762487852194 state 0x1 ref 1 00000100:00100000:2.0:1789760784.061959:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762488112610 time 762527248562 state 0x0 ref 1 00000100:00100000:2.0:1789760784.061973:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515810560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.062021:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25874432, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.062029:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.062045:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760784.062050:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.062056:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.062070:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.093110:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25874432 -> 25878528 00010000:00010000:2.0:1789760784.093132:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.093145:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.093160:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.093168:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.093173:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.093182:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.093201:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.093209:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d1ff25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.093311:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515810560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31329us (32172us total) trans 4294973634 rc 0/0 00000100:00100000:2.0:1789760784.093333:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6346 00000100:00100000:2.0:1789760784.093343:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762488112610 time 762527248562 state 0x2 ref 0 00000001:00080000:1.0:1789760784.093463:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973634 is committed 00000100:00100000:1.0:1789760784.097998:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760784.098092:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810688 00000100:00100000:0.0:1789760784.098102:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810688 00000100:00100000:0.0:1789760784.098156:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762488112610 time 762527248562 state 0x0 ref 1 00000100:00100000:0.0:1789760784.098171:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762527508978 time 762527248562 state 0x1 ref 1 00000100:00100000:0.0:1789760784.098236:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762527508978 time 762563524700 state 0x0 ref 1 00000100:00100000:0.0:1789760784.098248:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515810688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760784.098298:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25878528, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:0.0:1789760784.098478:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.098496:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760784.098501:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.098507:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.098521:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.127895:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973635 is committed 00002000:00010000:1.0:1789760784.127996:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25878528 -> 25882624 00010000:00010000:1.0:1789760784.128011:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.128023:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.128032:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.128039:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760784.128043:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.128052:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.128075:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.128083:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.128176:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515810688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29923us (30184us total) trans 4294973635 rc 0/0 00000100:00100000:1.0:1789760784.128186:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6347 00000100:00100000:1.0:1789760784.128196:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762527508978 time 762563524700 state 0x2 ref 0 00000100:00100000:3.0:1789760784.132856:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760784.133139:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810816 00000100:00100000:0.0:1789760784.133149:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810816 00000100:00100000:0.0:1789760784.133200:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762527508978 time 762563524700 state 0x0 ref 1 00000100:00100000:0.0:1789760784.133214:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762563785116 time 762563524700 state 0x1 ref 1 00000100:00100000:0.0:1789760784.133248:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762563785116 time 762598537920 state 0x0 ref 1 00000100:00100000:0.0:1789760784.133261:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515810816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760784.133428:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25882624, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:0.0:1789760784.133439:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.133455:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760784.133460:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.133466:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.133481:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760784.162248:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25882624 -> 25886720 00010000:00010000:3.0:1789760784.162268:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.162282:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.162292:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.162299:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760784.162314:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.162331:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.162351:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.162358:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d1ff33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.162450:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515810816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29187us (29597us total) trans 4294973636 rc 0/0 00000100:00100000:3.0:1789760784.162460:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6348 00000100:00100000:3.0:1789760784.162469:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762563785116 time 762598537920 state 0x2 ref 0 00000100:00100000:3.0:1789760784.166053:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.166148:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515810944 00000100:00100000:2.0:1789760784.166158:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515810944 00000100:00100000:2.0:1789760784.166211:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762563785116 time 762598537920 state 0x0 ref 1 00000100:00100000:2.0:1789760784.166230:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762598798336 time 762598537920 state 0x1 ref 1 00000100:00100000:2.0:1789760784.166263:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762598798336 time 762631552218 state 0x0 ref 1 00000100:00100000:2.0:1789760784.166275:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515810944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.166335:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25886720, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.166343:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.166359:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760784.166363:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.166369:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.166383:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.169972:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973636 is committed 00002000:00010000:3.0:1789760784.198745:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25886720 -> 25890816 00010000:00010000:3.0:1789760784.198766:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.198781:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.198790:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.198797:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760784.198802:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.198810:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.198829:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.198837:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d1ff3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.198934:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515810944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32655us (32885us total) trans 4294973637 rc 0/0 00000100:00100000:3.0:1789760784.198944:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6349 00000100:00100000:3.0:1789760784.198954:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762598798336 time 762631552218 state 0x2 ref 0 00000001:00080000:1.0:1789760784.199161:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973637 is committed 00000100:00100000:2.0:1789760784.204158:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.204253:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811072 00000100:00100000:3.0:1789760784.204263:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811072 00000100:00100000:3.0:1789760784.204315:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762598798336 time 762631552218 state 0x0 ref 1 00000100:00100000:3.0:1789760784.204400:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762631812634 time 762631552218 state 0x1 ref 1 00000100:00100000:3.0:1789760784.204469:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762631812634 time 762669758011 state 0x0 ref 1 00000100:00100000:3.0:1789760784.204483:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760784.204528:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25890816, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:3.0:1789760784.204537:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.204552:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760784.204557:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.204563:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.204576:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.238483:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973638 is committed 00002000:00010000:3.0:1789760784.238579:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25890816 -> 25894912 00010000:00010000:3.0:1789760784.238653:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.238675:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.238685:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.238692:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760784.238696:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.238705:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.238789:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.238797:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.238919:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34434us (34766us total) trans 4294973638 rc 0/0 00000100:00100000:3.0:1789760784.238929:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6350 00000100:00100000:3.0:1789760784.238938:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762631812634 time 762669758011 state 0x2 ref 0 00000100:00100000:2.0:1789760784.244204:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760784.244300:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811200 00000100:00100000:1.0:1789760784.244309:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811200 00000100:00100000:1.0:1789760784.244431:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762631812634 time 762669758011 state 0x0 ref 1 00000100:00100000:1.0:1789760784.244446:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762670018427 time 762669758011 state 0x1 ref 1 00000100:00100000:1.0:1789760784.244477:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762670018427 time 762709766028 state 0x0 ref 1 00000100:00100000:1.0:1789760784.244490:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760784.244539:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25894912, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:1.0:1789760784.244547:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.244573:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760784.244578:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.244585:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.244600:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.297838:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973639 is committed 00002000:00010000:0.0:1789760784.301125:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25894912 -> 25899008 00010000:00010000:0.0:1789760784.301496:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.301511:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.301521:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.301528:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760784.301534:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.301544:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.301565:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.301573:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.301925:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57431us (57725us total) trans 4294973639 rc 0/0 00000100:00100000:0.0:1789760784.301936:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6351 00000100:00100000:0.0:1789760784.301946:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762670018427 time 762709766028 state 0x2 ref 0 00000100:00100000:0.0:1789760784.308444:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.308518:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811328 00000100:00100000:3.0:1789760784.308527:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811328 00000100:00100000:3.0:1789760784.308690:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762670018427 time 762709766028 state 0x0 ref 1 00000100:00100000:3.0:1789760784.308835:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762710026444 time 762709766028 state 0x1 ref 1 00000100:00100000:3.0:1789760784.308908:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762710026444 time 762774196412 state 0x0 ref 1 00000100:00100000:3.0:1789760784.308923:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760784.308986:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25899008, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:3.0:1789760784.308996:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.309010:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760784.309014:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.309020:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.309032:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.345609:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973640 is committed 00002000:00010000:1.0:1789760784.345944:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25899008 -> 25903104 00010000:00010000:1.0:1789760784.345962:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.345976:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.345985:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.345993:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760784.345997:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.346007:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.346027:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.346036:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d1ff4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.346175:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37249us (37735us total) trans 4294973640 rc 0/0 00000100:00100000:1.0:1789760784.346185:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6352 00000100:00100000:1.0:1789760784.346196:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762710026444 time 762774196412 state 0x2 ref 0 00000100:00100000:2.0:1789760784.350641:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760784.350884:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811456 00000100:00100000:1.0:1789760784.350893:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811456 00000100:00100000:1.0:1789760784.350944:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762710026444 time 762774196412 state 0x0 ref 1 00000100:00100000:1.0:1789760784.350959:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762774456828 time 762774196412 state 0x1 ref 1 00000100:00100000:1.0:1789760784.350988:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762774456828 time 762816277638 state 0x0 ref 1 00000100:00100000:1.0:1789760784.351000:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760784.351057:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25903104, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:1.0:1789760784.351067:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.351082:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760784.351086:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.351093:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.351111:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.394491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973641 is committed 00002000:00010000:0.0:1789760784.394823:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25903104 -> 25907200 00010000:00010000:0.0:1789760784.394840:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.394853:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.394862:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.394875:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760784.394880:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.394888:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.394910:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.394918:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d1ff56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.395127:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44123us (44490us total) trans 4294973641 rc 0/0 00000100:00100000:0.0:1789760784.395137:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6353 00000100:00100000:0.0:1789760784.395147:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762774456828 time 762816277638 state 0x2 ref 0 00000100:00100000:2.0:1789760784.403124:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.403225:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811584 00000100:00100000:3.0:1789760784.403234:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811584 00000100:00100000:3.0:1789760784.403288:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762774456828 time 762816277638 state 0x0 ref 1 00000100:00100000:3.0:1789760784.403386:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762816538054 time 762816277638 state 0x1 ref 1 00000100:00100000:3.0:1789760784.403475:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762816538054 time 762868763372 state 0x0 ref 1 00000100:00100000:3.0:1789760784.403489:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760784.403540:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25907200, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:3.0:1789760784.403549:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.403566:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760784.403571:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.403578:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.403593:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.481155:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973642 is committed 00002000:00010000:2.0:1789760784.481359:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25907200 -> 25911296 00010000:00010000:2.0:1789760784.481375:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.481388:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.481396:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.481405:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.481411:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.481421:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.481441:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.481449:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d1ff5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.481541:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 78050us (78422us total) trans 4294973642 rc 0/0 00000100:00100000:2.0:1789760784.481551:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6354 00000100:00100000:2.0:1789760784.481561:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762816538054 time 762868763372 state 0x2 ref 0 00000100:00100000:3.0:1789760784.490870:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.491194:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811712 00000100:00100000:2.0:1789760784.491205:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811712 00000100:00100000:2.0:1789760784.491261:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762816538054 time 762868763372 state 0x0 ref 1 00000100:00100000:2.0:1789760784.491276:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762869023788 time 762868763372 state 0x1 ref 1 00000100:00100000:2.0:1789760784.491308:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762869023788 time 762956597726 state 0x0 ref 1 00000100:00100000:2.0:1789760784.491321:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515811712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.491369:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25911296, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.491377:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.491392:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760784.491396:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.491403:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.491415:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.521317:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973643 is committed 00002000:00010000:2.0:1789760784.521639:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25911296 -> 25915392 00010000:00010000:2.0:1789760784.521656:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.521679:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.521687:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.521695:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.521700:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.521744:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.521764:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.521771:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.521869:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515811712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30545us (31001us total) trans 4294973643 rc 0/0 00000100:00100000:2.0:1789760784.521880:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6355 00000100:00100000:2.0:1789760784.521890:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762869023788 time 762956597726 state 0x2 ref 0 00000100:00100000:1.0:1789760784.525579:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.526169:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811840 00000100:00100000:3.0:1789760784.526180:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811840 00000100:00100000:3.0:1789760784.526234:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762869023788 time 762956597726 state 0x0 ref 1 00000100:00100000:3.0:1789760784.526249:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762956858142 time 762956597726 state 0x1 ref 1 00000100:00100000:3.0:1789760784.526281:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762956858142 time 762991570708 state 0x0 ref 1 00000100:00100000:3.0:1789760784.526294:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515811840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760784.526342:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25915392, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:3.0:1789760784.526350:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.526366:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760784.526370:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.526376:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.526389:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.554321:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25915392 -> 25919488 00010000:00010000:2.0:1789760784.554341:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.554361:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.554370:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.554377:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.554383:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.554392:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.554412:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.554420:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d1ff6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.554512:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515811840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28216us (28937us total) trans 4294973644 rc 0/0 00000100:00100000:2.0:1789760784.554523:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6356 00000100:00100000:2.0:1789760784.554533:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762956858142 time 762991570708 state 0x2 ref 0 00000001:00080000:1.0:1789760784.554608:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973644 is committed 00000100:00100000:3.0:1789760784.558877:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760784.558937:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515811968 00000100:00100000:0.0:1789760784.558947:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515811968 00000100:00100000:0.0:1789760784.559005:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762956858142 time 762991570708 state 0x0 ref 1 00000100:00100000:0.0:1789760784.559019:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762991831124 time 762991570708 state 0x1 ref 1 00000100:00100000:0.0:1789760784.559080:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762991831124 time 763024368076 state 0x0 ref 1 00000100:00100000:0.0:1789760784.559104:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515811968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760784.559162:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25919488, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:0.0:1789760784.559171:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.559187:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.559192:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.559197:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.559211:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.593914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973645 is committed 00002000:00010000:2.0:1789760784.594014:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25919488 -> 25923584 00010000:00010000:2.0:1789760784.594029:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.594042:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.594051:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.594058:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.594063:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.594073:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.594092:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.594100:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d1ff72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.594447:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515811968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35339us (35572us total) trans 4294973645 rc 0/0 00000100:00100000:2.0:1789760784.594458:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6357 00000100:00100000:2.0:1789760784.594468:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762991831124 time 763024368076 state 0x2 ref 0 00000100:00100000:3.0:1789760784.601078:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760784.601192:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812096 00000100:00100000:1.0:1789760784.601204:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812096 00000100:00100000:1.0:1789760784.601258:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 762991831124 time 763024368076 state 0x0 ref 1 00000100:00100000:1.0:1789760784.601272:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763024628492 time 763024368076 state 0x1 ref 1 00000100:00100000:1.0:1789760784.601953:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763024628492 time 763066593155 state 0x0 ref 1 00000100:00100000:1.0:1789760784.601971:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515812096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760784.602022:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25923584, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:1.0:1789760784.602031:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.602047:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.602051:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.602058:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.602077:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760784.635852:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25923584 -> 25927680 00010000:00010000:3.0:1789760784.635872:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.635885:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.635894:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.635901:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760784.635906:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.635914:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.635933:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.635945:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d1ff79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.636038:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515812096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34067us (34964us total) trans 4294973646 rc 0/0 00000100:00100000:3.0:1789760784.636047:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6358 00000100:00100000:3.0:1789760784.636058:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763024628492 time 763066593155 state 0x2 ref 0 00000001:00080000:2.0:1789760784.638491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973646 is committed 00000100:00100000:1.0:1789760784.639633:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.639803:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812224 00000100:00100000:2.0:1789760784.639813:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812224 00000100:00100000:2.0:1789760784.639863:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763024628492 time 763066593155 state 0x0 ref 1 00000100:00100000:2.0:1789760784.639877:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763066853571 time 763066593155 state 0x1 ref 1 00000100:00100000:2.0:1789760784.639905:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763066853571 time 763105195032 state 0x0 ref 1 00000100:00100000:2.0:1789760784.639917:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515812224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.639960:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25927680, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.639968:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.639982:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.639986:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.639992:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.640005:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760784.670447:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25927680 -> 25931776 00010000:00010000:3.0:1789760784.670467:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.670479:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.670488:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.670495:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760784.670500:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.670509:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.670533:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.670540:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d1ff80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.670734:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515812224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30787us (31077us total) trans 4294973647 rc 0/0 00000100:00100000:3.0:1789760784.670746:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6359 00000100:00100000:3.0:1789760784.670755:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763066853571 time 763105195032 state 0x2 ref 0 00000001:00080000:1.0:1789760784.673953:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973647 is committed 00000100:00100000:1.0:1789760784.674436:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.674544:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812352 00000100:00100000:2.0:1789760784.674555:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812352 00000100:00100000:2.0:1789760784.674673:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763066853571 time 763105195032 state 0x0 ref 1 00000100:00100000:2.0:1789760784.674688:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763105455448 time 763105195032 state 0x1 ref 1 00000100:00100000:2.0:1789760784.674841:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763105455448 time 763140128452 state 0x0 ref 1 00000100:00100000:2.0:1789760784.674855:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515812352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.674902:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25931776, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.674910:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.674926:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.674931:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.674937:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.674951:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.701832:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760784.704817:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973648 is committed 00002000:00010000:2.0:1789760784.704917:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25931776 -> 25935872 00010000:00010000:2.0:1789760784.704931:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.704944:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.704954:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.704961:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.705026:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.705036:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.705055:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.705063:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.705147:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515812352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30290us (30710us total) trans 4294973648 rc 0/0 00000100:00100000:2.0:1789760784.705157:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6360 00000100:00100000:2.0:1789760784.705166:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763105455448 time 763140128452 state 0x2 ref 0 00000100:00100000:2.0:1789760784.709249:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760784.709410:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812480 00000100:00100000:1.0:1789760784.709419:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812480 00000100:00100000:1.0:1789760784.709468:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763105455448 time 763140128452 state 0x0 ref 1 00000100:00100000:1.0:1789760784.709482:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763140388868 time 763140128452 state 0x1 ref 1 00000100:00100000:1.0:1789760784.709509:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763140388868 time 763174799325 state 0x0 ref 1 00000100:00100000:1.0:1789760784.709520:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515812480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760784.709564:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25935872, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:1.0:1789760784.709573:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.709599:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.709603:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.709609:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.709628:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.735750:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25935872 -> 25939968 00010000:00010000:1.0:1789760784.735771:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.735785:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.735795:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.735802:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760784.735808:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.735817:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.735834:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.735843:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ff8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.735946:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515812480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26421us (26700us total) trans 4294973649 rc 0/0 00000100:00100000:1.0:1789760784.735957:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6361 00000100:00100000:1.0:1789760784.735977:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763140388868 time 763174799325 state 0x2 ref 0 00000001:00080000:3.0:1789760784.736110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973649 is committed 00000100:00100000:1.0:1789760784.741692:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.741746:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812608 00000100:00100000:3.0:1789760784.741756:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812608 00000100:00100000:3.0:1789760784.741811:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763140388868 time 763174799325 state 0x0 ref 1 00000100:00100000:3.0:1789760784.741828:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763175059741 time 763174799325 state 0x1 ref 1 00000100:00100000:0.0:1789760784.742057:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763175059741 time 763207344068 state 0x0 ref 1 00000100:00100000:0.0:1789760784.742073:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515812608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760784.742122:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25939968, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:0.0:1789760784.742131:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.742147:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.742152:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.742157:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.742175:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.787956:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25939968 -> 25944064 00010000:00010000:1.0:1789760784.787976:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.787989:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.787999:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.788006:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760784.788011:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.788019:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.788037:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.788049:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d1ff95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.788142:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515812608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46068us (46453us total) trans 4294973650 rc 0/0 00000100:00100000:1.0:1789760784.788153:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6362 00000100:00100000:1.0:1789760784.788163:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763175059741 time 763207344068 state 0x2 ref 0 00000001:00080000:3.0:1789760784.788246:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973650 is committed 00000100:00100000:0.0:1789760784.792225:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760784.792387:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812736 00000100:00100000:2.0:1789760784.792397:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812736 00000100:00100000:2.0:1789760784.792458:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763175059741 time 763207344068 state 0x0 ref 1 00000100:00100000:2.0:1789760784.792475:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763207604484 time 763207344068 state 0x1 ref 1 00000100:00100000:2.0:1789760784.792508:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763207604484 time 763257797184 state 0x0 ref 1 00000100:00100000:2.0:1789760784.792520:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515812736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760784.792572:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25944064, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:2.0:1789760784.792592:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.792609:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.792613:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.792619:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.792632:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.835323:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25944064 -> 25948160 00010000:00010000:2.0:1789760784.835344:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.835358:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.835367:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.835374:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760784.835379:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.835387:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.835407:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.835415:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d1ff9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.835502:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515812736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42979us (43282us total) trans 4294973651 rc 0/0 00000100:00100000:2.0:1789760784.835513:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6363 00000100:00100000:2.0:1789760784.835523:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763207604484 time 763257797184 state 0x2 ref 0 00000001:00080000:2.0:1789760784.835682:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973651 is committed 00000100:00100000:2.0:1789760784.839108:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.839182:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812864 00000100:00100000:3.0:1789760784.839193:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812864 00000100:00100000:3.0:1789760784.839241:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763207604484 time 763257797184 state 0x0 ref 1 00000100:00100000:3.0:1789760784.839256:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763258057600 time 763257797184 state 0x1 ref 1 00000100:00100000:3.0:1789760784.839285:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763258057600 time 763304574494 state 0x0 ref 1 00000100:00100000:3.0:1789760784.839300:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515812864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760784.839351:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25948160, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc807 00010000:00010000:3.0:1789760784.839360:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.839375:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.839379:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.839385:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.839398:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.869062:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25948160 -> 25952256 00002000:00010000:1.0:1789760784.869070:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:1.0:1789760784.869073:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00010000:00010000:1.0:1789760784.869086:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.869100:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.869109:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.869116:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760784.869121:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.869131:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.869152:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.869160:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d1ffa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.869292:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973652 is committed 00000100:00100000:1.0:1789760784.869370:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515812864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30066us (30268us total) trans 4294973652 rc 0/0 00000100:00100000:1.0:1789760784.869382:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6364 00000100:00100000:1.0:1789760784.869393:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763258057600 time 763304574494 state 0x2 ref 0 00000100:00100000:2.0:1789760784.872168:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760784.872518:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515812992 00000100:00100000:3.0:1789760784.872528:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515812992 00000100:00100000:3.0:1789760784.872638:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763258057600 time 763304574494 state 0x0 ref 1 00000100:00100000:3.0:1789760784.872654:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763304834910 time 763304574494 state 0x1 ref 1 00000100:00100000:3.0:1789760784.872740:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763304834910 time 763338028823 state 0x0 ref 1 00000100:00100000:3.0:1789760784.872749:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515812992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760784.872786:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25952256, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760784.872793:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.872809:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.872812:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.872816:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.872826:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.894727:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544420864:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760784.894799:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.894855:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420864 00000100:00100000:1.0:1789760784.894865:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420864 00000100:00080000:2.0:1789760784.895081:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760784.895549:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1789760784.895561:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544420864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760784.895585:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3832545280 avail 3794796544 00000100:00100000:2.0:1789760784.895632:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8063b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544420992:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760784.895644:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544420864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (845us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.895650:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1789760784.895675:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.895689:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544420992 00000100:00100000:1.0:1789760784.895693:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544420992 00000100:00100000:1.0:1789760784.895737:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789760784.895743:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544420992:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760784.895743:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544420864:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760784.895762:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00000024:1.0:1789760784.895765:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760784.895803:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544420992:12345-0@lo:13:osp-pre-1-0.0 Request processed in 60us (128us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.895810:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789760784.895851:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8063b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544420992:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760784.903879:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973653 is committed 00002000:00010000:0.0:1789760784.903980:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25952256 -> 25956352 00010000:00010000:0.0:1789760784.903996:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.904010:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.904020:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.904028:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760784.904033:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.904043:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.904062:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.904071:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d1ffaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.904220:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515812992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31467us (32055us total) trans 4294973653 rc 0/0 00000100:00100000:0.0:1789760784.904231:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6365 00000100:00100000:0.0:1789760784.904240:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763304834910 time 763338028823 state 0x2 ref 0 00000100:00100000:2.0:1789760784.913168:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760784.913759:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813120 00000100:00100000:1.0:1789760784.913769:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813120 00000100:00100000:1.0:1789760784.913825:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763304834910 time 763338028823 state 0x0 ref 1 00000100:00100000:1.0:1789760784.913840:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763338289239 time 763338028823 state 0x1 ref 1 00000100:00100000:1.0:1789760784.913868:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763338289239 time 763379157895 state 0x0 ref 1 00000100:00100000:1.0:1789760784.913879:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760784.913931:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25956352, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:1.0:1789760784.913939:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.913954:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.913957:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.913963:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.913975:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.958300:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25956352 -> 25960448 00010000:00010000:0.0:1789760784.958320:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.958334:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.958343:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.958351:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760784.958356:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.958365:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.958384:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.958398:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d1ffb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.958492:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515813120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44610us (45331us total) trans 4294973654 rc 0/0 00000100:00100000:0.0:1789760784.958503:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6366 00000100:00100000:0.0:1789760784.958512:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763338289239 time 763379157895 state 0x2 ref 0 00000001:00080000:1.0:1789760784.959197:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973654 is committed 00000100:00100000:2.0:1789760784.969290:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760784.973039:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813248 00000100:00100000:1.0:1789760784.973054:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813248 00000100:00100000:1.0:1789760784.973117:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763338289239 time 763379157895 state 0x0 ref 1 00000100:00100000:1.0:1789760784.973134:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763379418311 time 763379157895 state 0x1 ref 1 00000100:00100000:1.0:1789760784.973170:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763379418311 time 763438458875 state 0x0 ref 1 00000100:00100000:1.0:1789760784.973183:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515813248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760784.973233:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25960448, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:1.0:1789760784.973242:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.973257:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760784.973262:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.973268:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.973281:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.019762:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973655 is committed 00002000:00010000:0.0:1789760785.019919:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25960448 -> 25964544 00010000:00010000:0.0:1789760785.019936:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.019948:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.019957:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.019964:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.019969:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.019978:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.020001:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.020009:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.020098:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515813248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46912us (50812us total) trans 4294973655 rc 0/0 00000100:00100000:0.0:1789760785.020109:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6367 00000100:00100000:0.0:1789760785.020119:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763379418311 time 763438458875 state 0x2 ref 0 00000100:00100000:3.0:1789760785.025562:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760785.027670:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813376 00000100:00100000:1.0:1789760785.027779:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813376 00000100:00100000:1.0:1789760785.027835:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763379418311 time 763438458875 state 0x0 ref 1 00000100:00100000:1.0:1789760785.027850:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763438719291 time 763438458875 state 0x1 ref 1 00000100:00100000:1.0:1789760785.027919:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763438719291 time 763493207562 state 0x0 ref 1 00000100:00100000:1.0:1789760785.027934:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760785.027979:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25964544, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:1.0:1789760785.027988:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.028011:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760785.028017:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.028023:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.028037:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.090632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973656 is committed 00002000:00010000:2.0:1789760785.090804:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25964544 -> 25968640 00010000:00010000:2.0:1789760785.090821:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.090835:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.090845:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.090851:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760785.090856:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.090866:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.090887:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.090895:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.091163:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63224us (65604us total) trans 4294973656 rc 0/0 00000100:00100000:2.0:1789760785.091173:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6368 00000100:00100000:2.0:1789760785.091184:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763438719291 time 763493207562 state 0x2 ref 0 00000100:00100000:2.0:1789760785.098952:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.099045:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813504 00000100:00100000:0.0:1789760785.099055:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813504 00000100:00100000:0.0:1789760785.099111:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763438719291 time 763493207562 state 0x0 ref 1 00000100:00100000:0.0:1789760785.099127:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763493467978 time 763493207562 state 0x1 ref 1 00000100:00100000:0.0:1789760785.099158:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763493467978 time 763564447099 state 0x0 ref 1 00000100:00100000:0.0:1789760785.099169:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515813504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.099219:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25968640, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.099228:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.099244:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760785.099249:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.099255:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.099281:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.143903:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973657 is committed 00002000:00010000:0.0:1789760785.143999:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25968640 -> 25972736 00010000:00010000:0.0:1789760785.144014:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.144028:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.144037:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.144045:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.144050:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.144059:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.144079:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.144086:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.144383:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515813504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45210us (45436us total) trans 4294973657 rc 0/0 00000100:00100000:0.0:1789760785.144394:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6369 00000100:00100000:0.0:1789760785.144403:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763493467978 time 763564447099 state 0x2 ref 0 00000100:00100000:0.0:1789760785.152504:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.152687:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813632 00000100:00100000:3.0:1789760785.152699:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813632 00000100:00100000:3.0:1789760785.152790:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763493467978 time 763564447099 state 0x0 ref 1 00000100:00100000:3.0:1789760785.152805:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763564707515 time 763564447099 state 0x1 ref 1 00000100:00100000:3.0:1789760785.152824:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763564707515 time 763618113696 state 0x0 ref 1 00000100:00100000:3.0:1789760785.152835:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.152879:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25972736, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.152888:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.152904:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760785.152909:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.152914:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.152927:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.197241:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973658 is committed 00002000:00010000:3.0:1789760785.197418:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25972736 -> 25976832 00010000:00010000:3.0:1789760785.197431:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.197441:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.197447:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.197451:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.197454:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.197460:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.197475:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.197479:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.197571:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44732us (45072us total) trans 4294973658 rc 0/0 00000100:00100000:3.0:1789760785.198837:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6370 00000100:00100000:3.0:1789760785.198847:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763564707515 time 763618113696 state 0x2 ref 0 00000100:00100000:2.0:1789760785.202002:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.202063:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813760 00000100:00100000:0.0:1789760785.202074:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813760 00000100:00100000:0.0:1789760785.202126:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763564707515 time 763618113696 state 0x0 ref 1 00000100:00100000:0.0:1789760785.202142:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763618374112 time 763618113696 state 0x1 ref 1 00000100:00100000:0.0:1789760785.202170:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763618374112 time 763667459272 state 0x0 ref 1 00000100:00100000:0.0:1789760785.202183:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.202242:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25976832, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.202252:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.202275:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760785.202279:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.202286:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.202306:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.235363:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973659 is committed 00002000:00010000:0.0:1789760785.235433:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25976832 -> 25980928 00010000:00010000:0.0:1789760785.235448:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.235461:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.235470:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.235477:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.235483:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.235492:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.235513:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.235521:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1ffd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.235935:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515813760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33749us (33936us total) trans 4294973659 rc 0/0 00000100:00100000:0.0:1789760785.235946:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6371 00000100:00100000:0.0:1789760785.235956:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763618374112 time 763667459272 state 0x2 ref 0 00000100:00100000:2.0:1789760785.240289:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760785.240474:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515813888 00000100:00100000:1.0:1789760785.240484:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515813888 00000100:00100000:1.0:1789760785.240537:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763618374112 time 763667459272 state 0x0 ref 1 00000100:00100000:1.0:1789760785.240553:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763667719688 time 763667459272 state 0x1 ref 1 00000100:00100000:1.0:1789760785.240681:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763667719688 time 763705967999 state 0x0 ref 1 00000100:00100000:1.0:1789760785.240695:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515813888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760785.240807:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25980928, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:1.0:1789760785.240817:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.240837:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760785.240841:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.240847:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.240860:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760785.284316:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25980928 -> 25985024 00010000:00010000:3.0:1789760785.284337:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.284351:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.284360:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.284366:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.284371:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.284381:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.284401:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.284415:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d1ffdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.284512:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515813888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43815us (44229us total) trans 4294973660 rc 0/0 00000100:00100000:3.0:1789760785.284522:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6372 00000100:00100000:3.0:1789760785.284532:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763667719688 time 763705967999 state 0x2 ref 0 00000001:00080000:0.0:1789760785.286160:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973660 is committed 00000100:00100000:0.0:1789760785.290791:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760785.290851:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814016 00000100:00100000:1.0:1789760785.290862:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814016 00000100:00100000:1.0:1789760785.290912:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763667719688 time 763705967999 state 0x0 ref 1 00000100:00100000:1.0:1789760785.290927:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763706228415 time 763705967999 state 0x1 ref 1 00000100:00100000:1.0:1789760785.290958:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763706228415 time 763756246532 state 0x0 ref 1 00000100:00100000:1.0:1789760785.290971:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515814016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760785.291017:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25985024, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:1.0:1789760785.291026:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.291041:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760785.291046:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.291052:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.291072:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.325759:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973661 is committed 00002000:00010000:3.0:1789760785.325941:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25985024 -> 25989120 00010000:00010000:3.0:1789760785.325957:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.325971:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.325981:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.325989:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.325994:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.326004:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.326029:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.326038:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d1ffe2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.326164:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515814016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35191us (35378us total) trans 4294973661 rc 0/0 00000100:00100000:3.0:1789760785.326175:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6373 00000100:00100000:3.0:1789760785.326185:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763706228415 time 763756246532 state 0x2 ref 0 00000100:00100000:2.0:1789760785.333459:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.333647:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814144 00000100:00100000:3.0:1789760785.333668:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814144 00000100:00100000:3.0:1789760785.333741:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763706228415 time 763756246532 state 0x0 ref 1 00000100:00100000:3.0:1789760785.333756:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763756506948 time 763756246532 state 0x1 ref 1 00000100:00100000:3.0:1789760785.333826:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763756506948 time 763799114195 state 0x0 ref 1 00000100:00100000:3.0:1789760785.333839:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515814144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.333888:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25989120, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.333899:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.333915:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760785.333921:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.333927:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.333941:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760785.362167:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25989120 -> 25993216 00010000:00010000:3.0:1789760785.362187:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.362201:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.362210:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.362217:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.362222:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.362231:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.362249:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.362256:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1ffe9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.362352:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515814144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28510us (28899us total) trans 4294973662 rc 0/0 00000100:00100000:3.0:1789760785.362362:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6374 00000100:00100000:3.0:1789760785.362371:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763756506948 time 763799114195 state 0x2 ref 0 00000001:00080000:0.0:1789760785.364063:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973662 is committed 00000100:00100000:2.0:1789760785.366177:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.366410:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814272 00000100:00100000:0.0:1789760785.366421:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814272 00000100:00100000:0.0:1789760785.366473:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763756506948 time 763799114195 state 0x0 ref 1 00000100:00100000:0.0:1789760785.366488:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763799374611 time 763799114195 state 0x1 ref 1 00000100:00100000:0.0:1789760785.366517:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763799374611 time 763831806731 state 0x0 ref 1 00000100:00100000:0.0:1789760785.366531:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515814272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.366590:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25993216, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.366599:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.366616:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760785.366620:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.366626:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.366642:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.403734:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973663 is committed 00002000:00010000:0.0:1789760785.403833:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25993216 -> 25997312 00010000:00010000:0.0:1789760785.403848:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.403859:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.403868:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.403876:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.403881:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.403889:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.403906:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.403913:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fff0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.404010:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515814272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37477us (37838us total) trans 4294973663 rc 0/0 00000100:00100000:0.0:1789760785.404020:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6375 00000100:00100000:0.0:1789760785.404029:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763799374611 time 763831806731 state 0x2 ref 0 00000100:00100000:1.0:1789760785.410796:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.410931:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814400 00000100:00100000:3.0:1789760785.410942:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814400 00000100:00100000:3.0:1789760785.410998:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763799374611 time 763831806731 state 0x0 ref 1 00000100:00100000:3.0:1789760785.411018:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763832067147 time 763831806731 state 0x1 ref 1 00000100:00100000:3.0:1789760785.411047:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763832067147 time 763876336406 state 0x0 ref 1 00000100:00100000:3.0:1789760785.411058:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515814400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.411106:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 25997312, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.411114:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.411128:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760785.411139:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.411144:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.411157:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.445964:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973664 is committed 00002000:00010000:3.0:1789760785.446028:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 25997312 -> 26001408 00010000:00010000:3.0:1789760785.446042:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.446055:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.446064:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.446071:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.446076:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.446085:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.446105:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.446112:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d1fff7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.446197:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515814400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35135us (35405us total) trans 4294973664 rc 0/0 00000100:00100000:3.0:1789760785.446207:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6376 00000100:00100000:3.0:1789760785.446216:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763832067147 time 763876336406 state 0x2 ref 0 00000100:00100000:2.0:1789760785.449982:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.450087:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814528 00000100:00100000:0.0:1789760785.450097:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814528 00000100:00100000:0.0:1789760785.450147:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763832067147 time 763876336406 state 0x0 ref 1 00000100:00100000:0.0:1789760785.450162:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763876596822 time 763876336406 state 0x1 ref 1 00000100:00100000:0.0:1789760785.450193:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763876596822 time 763915482518 state 0x0 ref 1 00000100:00100000:0.0:1789760785.450204:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515814528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.450251:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26001408, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.450260:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.450275:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760785.450279:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.450285:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.450298:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.482012:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973665 is committed 00002000:00010000:1.0:1789760785.482178:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26001408 -> 26005504 00010000:00010000:1.0:1789760785.482193:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.482206:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.482216:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.482224:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760785.482229:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.482239:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.482259:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.482267:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d1fffe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.482433:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515814528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32225us (32455us total) trans 4294973665 rc 0/0 00000100:00100000:1.0:1789760785.482444:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6377 00000100:00100000:1.0:1789760785.482453:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763876596822 time 763915482518 state 0x2 ref 0 00000100:00100000:2.0:1789760785.486290:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.486447:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814656 00000100:00100000:3.0:1789760785.486458:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814656 00000100:00100000:3.0:1789760785.486510:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763876596822 time 763915482518 state 0x0 ref 1 00000100:00100000:3.0:1789760785.486525:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763915742934 time 763915482518 state 0x1 ref 1 00000100:00100000:3.0:1789760785.486555:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763915742934 time 763951844318 state 0x0 ref 1 00000100:00100000:3.0:1789760785.486567:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515814656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.486823:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26005504, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.486832:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.486847:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760785.486851:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.486857:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.486870:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760785.523979:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26005504 -> 26009600 00010000:00010000:3.0:1789760785.524001:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.524015:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.524025:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.524032:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.524037:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.524045:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.524064:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.524071:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20005 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.524167:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515814656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37597us (37880us total) trans 4294973666 rc 0/0 00000100:00100000:3.0:1789760785.524177:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6378 00000100:00100000:3.0:1789760785.524186:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763915742934 time 763951844318 state 0x2 ref 0 00000100:00100000:0.0:1789760785.528007:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.528088:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814784 00000100:00100000:3.0:1789760785.528099:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814784 00000100:00100000:3.0:1789760785.528153:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763915742934 time 763951844318 state 0x0 ref 1 00000100:00100000:3.0:1789760785.528175:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763952104734 time 763951844318 state 0x1 ref 1 00000100:00100000:3.0:1789760785.528207:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763952104734 time 763993495618 state 0x0 ref 1 00000100:00100000:3.0:1789760785.528219:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515814784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.528266:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26009600, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.528275:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.528291:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760785.528305:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.528312:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.528326:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.528987:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973666 is committed 00000001:00080000:0.0:1789760785.560956:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973667 is committed 00002000:00010000:3.0:1789760785.561077:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26009600 -> 26013696 00010000:00010000:3.0:1789760785.561094:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.561108:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.561123:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.561131:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.561137:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.561146:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.561167:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.561175:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d2000c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.561276:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515814784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33053us (33272us total) trans 4294973667 rc 0/0 00000100:00100000:3.0:1789760785.561286:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6379 00000100:00100000:3.0:1789760785.561306:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763952104734 time 763993495618 state 0x2 ref 0 00000100:00100000:2.0:1789760785.565905:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.565993:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515814912 00000100:00100000:3.0:1789760785.566005:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515814912 00000100:00100000:3.0:1789760785.566067:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763952104734 time 763993495618 state 0x0 ref 1 00000100:00100000:3.0:1789760785.566084:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763993756034 time 763993495618 state 0x1 ref 1 00000100:00100000:3.0:1789760785.566120:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763993756034 time 764031408681 state 0x0 ref 1 00000100:00100000:3.0:1789760785.566132:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515814912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.566188:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26013696, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.566197:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.566214:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.566218:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.566225:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.566238:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.601005:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973668 is committed 00002000:00010000:3.0:1789760785.601112:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26013696 -> 26017792 00010000:00010000:3.0:1789760785.601128:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.601141:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.601150:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.601158:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.601164:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.601173:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.601193:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.601200:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20013 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.601401:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515814912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35265us (35502us total) trans 4294973668 rc 0/0 00000100:00100000:3.0:1789760785.601414:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6380 00000100:00100000:3.0:1789760785.601423:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763993756034 time 764031408681 state 0x2 ref 0 00000100:00100000:0.0:1789760785.605205:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760785.605362:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815040 00000100:00100000:2.0:1789760785.605371:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815040 00000100:00100000:2.0:1789760785.605424:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 763993756034 time 764031408681 state 0x0 ref 1 00000100:00100000:2.0:1789760785.605440:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764031669097 time 764031408681 state 0x1 ref 1 00000100:00100000:2.0:1789760785.605502:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764031669097 time 764070790448 state 0x0 ref 1 00000100:00100000:2.0:1789760785.605514:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515815040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760785.605563:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26017792, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:2.0:1789760785.605584:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.605600:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.605605:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.605611:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.605623:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.641186:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973669 is committed 00002000:00010000:0.0:1789760785.641288:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26017792 -> 26021888 00010000:00010000:0.0:1789760785.641395:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.641409:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.641419:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.641427:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.641432:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.641441:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.641462:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.641471:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d2001a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.641915:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515815040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36396us (36712us total) trans 4294973669 rc 0/0 00000100:00100000:0.0:1789760785.641927:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6381 00000100:00100000:0.0:1789760785.641936:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764031669097 time 764070790448 state 0x2 ref 0 00000100:00100000:2.0:1789760785.645398:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.645479:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815168 00000100:00100000:0.0:1789760785.645489:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815168 00000100:00100000:0.0:1789760785.645539:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764031669097 time 764070790448 state 0x0 ref 1 00000100:00100000:0.0:1789760785.645554:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764071050864 time 764070790448 state 0x1 ref 1 00000100:00100000:0.0:1789760785.645594:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764071050864 time 764110883605 state 0x0 ref 1 00000100:00100000:0.0:1789760785.645605:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515815168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.645664:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26021888, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.645672:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.645688:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.645691:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.645697:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.645740:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760785.674487:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26021888 -> 26025984 00010000:00010000:0.0:1789760785.674509:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.674523:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.674532:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.674539:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.674544:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.674553:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.674585:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.674594:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20021 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.674697:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515815168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29088us (29302us total) trans 4294973670 rc 0/0 00000100:00100000:0.0:1789760785.674738:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6382 00000100:00100000:0.0:1789760785.674748:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764071050864 time 764110883605 state 0x2 ref 0 00000001:00080000:2.0:1789760785.674825:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973670 is committed 00000100:00100000:2.0:1789760785.678508:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760785.678671:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815296 00000100:00100000:1.0:1789760785.678681:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815296 00000100:00100000:1.0:1789760785.678890:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764071050864 time 764110883605 state 0x0 ref 1 00000100:00100000:1.0:1789760785.678905:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764111144021 time 764110883605 state 0x1 ref 1 00000100:00100000:1.0:1789760785.678975:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764111144021 time 764144263795 state 0x0 ref 1 00000100:00100000:1.0:1789760785.678990:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515815296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760785.679037:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26025984, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:1.0:1789760785.679048:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.679065:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760785.679070:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.679076:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.679090:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.706507:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973671 is committed 00002000:00010000:2.0:1789760785.706849:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26025984 -> 26030080 00010000:00010000:2.0:1789760785.706864:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.706877:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.706886:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.706893:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760785.706897:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.706906:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.706924:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.706933:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20028 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.707168:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515815296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28176us (28663us total) trans 4294973671 rc 0/0 00000100:00100000:2.0:1789760785.707179:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6383 00000100:00100000:2.0:1789760785.707189:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764111144021 time 764144263795 state 0x2 ref 0 00000100:00100000:1.0:1789760785.710963:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.711144:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815424 00000100:00100000:3.0:1789760785.711153:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815424 00000100:00100000:3.0:1789760785.711208:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764111144021 time 764144263795 state 0x0 ref 1 00000100:00100000:3.0:1789760785.711223:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764144524211 time 764144263795 state 0x1 ref 1 00000100:00100000:3.0:1789760785.711254:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764144524211 time 764176543732 state 0x0 ref 1 00000100:00100000:3.0:1789760785.711267:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515815424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.711422:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26030080, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.711432:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.711448:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.711452:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.711458:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.711472:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.728212:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760785.745320:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973672 is committed 00002000:00010000:3.0:1789760785.745481:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26030080 -> 26034176 00010000:00010000:3.0:1789760785.745496:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.745511:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.745520:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.745526:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.745530:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.745545:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.745567:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.745586:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2002f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.745682:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515815424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34413us (34724us total) trans 4294973672 rc 0/0 00000100:00100000:3.0:1789760785.745693:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6384 00000100:00100000:3.0:1789760785.745702:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764144524211 time 764176543732 state 0x2 ref 0 00000100:00100000:3.0:1789760785.749047:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.749144:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815552 00000100:00100000:0.0:1789760785.749153:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815552 00000100:00100000:0.0:1789760785.749205:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764144524211 time 764176543732 state 0x0 ref 1 00000100:00100000:0.0:1789760785.749219:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764176804148 time 764176543732 state 0x1 ref 1 00000100:00100000:0.0:1789760785.749250:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764176804148 time 764214539450 state 0x0 ref 1 00000100:00100000:0.0:1789760785.749263:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515815552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.749321:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26034176, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.749330:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.749346:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.749350:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.749356:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.749369:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.778192:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973673 is committed 00002000:00010000:0.0:1789760785.778290:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26034176 -> 26038272 00010000:00010000:0.0:1789760785.778316:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.778329:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.778338:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.778345:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.778350:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.778360:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.778380:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.778390:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20036 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.778482:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515815552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29216us (29438us total) trans 4294973673 rc 0/0 00000100:00100000:0.0:1789760785.778492:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6385 00000100:00100000:0.0:1789760785.778502:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764176804148 time 764214539450 state 0x2 ref 0 00000100:00100000:2.0:1789760785.782997:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.783451:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815680 00000100:00100000:3.0:1789760785.783461:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815680 00000100:00100000:3.0:1789760785.783515:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764176804148 time 764214539450 state 0x0 ref 1 00000100:00100000:3.0:1789760785.783530:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764214799866 time 764214539450 state 0x1 ref 1 00000100:00100000:3.0:1789760785.783562:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764214799866 time 764248851745 state 0x0 ref 1 00000100:00100000:3.0:1789760785.783585:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515815680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.783632:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26038272, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:3.0:1789760785.783641:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.783663:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.783667:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.783674:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.783686:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.815993:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973674 is committed 00002000:00010000:0.0:1789760785.816101:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26038272 -> 26042368 00010000:00010000:0.0:1789760785.816117:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.816130:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.816138:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.816146:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760785.816151:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.816161:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.816181:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.816189:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2003d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.816282:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515815680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32706us (33290us total) trans 4294973674 rc 0/0 00000100:00100000:0.0:1789760785.816303:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6386 00000100:00100000:0.0:1789760785.816313:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764214799866 time 764248851745 state 0x2 ref 0 00000100:00100000:2.0:1789760785.820153:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.820235:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815808 00000100:00100000:0.0:1789760785.820246:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815808 00000100:00100000:0.0:1789760785.820318:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764214799866 time 764248851745 state 0x0 ref 1 00000100:00100000:0.0:1789760785.820334:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764249112161 time 764248851745 state 0x1 ref 1 00000100:00100000:0.0:1789760785.820367:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764249112161 time 764285655210 state 0x0 ref 1 00000100:00100000:0.0:1789760785.820381:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515815808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.820438:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26042368, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc807 00010000:00010000:0.0:1789760785.820447:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.820463:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.820469:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.820475:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.820491:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.848028:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973675 is committed 00002000:00010000:2.0:1789760785.848132:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26042368 -> 26046464 00002000:00010000:2.0:1789760785.848137:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:2.0:1789760785.848140:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00010000:00010000:2.0:1789760785.848154:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.848166:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.848175:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.848182:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760785.848188:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.848197:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.848214:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.848222:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20044 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.848444:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515815808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28057us (28295us total) trans 4294973675 rc 0/0 00000100:00100000:2.0:1789760785.848459:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6387 00000100:00100000:2.0:1789760785.848469:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764249112161 time 764285655210 state 0x2 ref 0 00000100:00100000:3.0:1789760785.851349:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760785.851677:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515815936 00000100:00100000:0.0:1789760785.851687:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515815936 00000100:00100000:0.0:1789760785.851773:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764249112161 time 764285655210 state 0x0 ref 1 00000100:00100000:0.0:1789760785.851789:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764285915626 time 764285655210 state 0x1 ref 1 00000100:00100000:0.0:1789760785.851817:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764285915626 time 764317106263 state 0x0 ref 1 00000100:00100000:0.0:1789760785.851830:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515815936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760785.851876:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26046464, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:0.0:1789760785.851885:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.851900:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.851906:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.851912:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.851926:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.882898:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973676 is committed 00002000:00010000:3.0:1789760785.883021:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26046464 -> 26050560 00010000:00010000:3.0:1789760785.883038:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.883051:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.883061:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.883068:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.883073:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.883086:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.883106:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.883113:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2004b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.887666:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515815936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35815us (36303us total) trans 4294973676 rc 0/0 00000100:00100000:3.0:1789760785.887682:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6388 00000100:00100000:3.0:1789760785.887692:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764285915626 time 764317106263 state 0x2 ref 0 00000100:00100000:3.0:1789760785.893293:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760785.894196:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816064 00000100:00100000:2.0:1789760785.894208:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816064 00000100:00100000:2.0:1789760785.894280:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764285915626 time 764317106263 state 0x0 ref 1 00000100:00100000:2.0:1789760785.894302:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764317366679 time 764317106263 state 0x1 ref 1 00000100:00100000:2.0:1789760785.894333:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764317366679 time 764359622142 state 0x0 ref 1 00000100:00100000:2.0:1789760785.894345:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760785.894394:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26050560, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:2.0:1789760785.894404:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.894419:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.894424:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.894434:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.894453:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.950863:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973677 is committed 00002000:00010000:3.0:1789760785.950981:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26050560 -> 26054656 00010000:00010000:3.0:1789760785.950997:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.951009:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.951018:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.951031:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760785.951036:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.951046:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.951065:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.951072:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20052 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.951179:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56831us (57891us total) trans 4294973677 rc 0/0 00000100:00100000:3.0:1789760785.951191:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6389 00000100:00100000:3.0:1789760785.951201:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764317366679 time 764359622142 state 0x2 ref 0 00000100:00100000:1.0:1789760785.956115:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760785.956197:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816192 00000100:00100000:3.0:1789760785.956207:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816192 00000100:00100000:3.0:1789760785.956263:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764317366679 time 764359622142 state 0x0 ref 1 00000100:00100000:3.0:1789760785.956278:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764359882558 time 764359622142 state 0x1 ref 1 00000100:00100000:3.0:1789760785.956317:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764359882558 time 764421605548 state 0x0 ref 1 00000100:00100000:3.0:1789760785.956335:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760785.956382:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26054656, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760785.956390:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.956405:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.956410:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.956416:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.956429:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.017526:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26054656 -> 26058752 00010000:00010000:1.0:1789760786.017545:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.017559:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.017582:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.017590:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760786.017595:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.017604:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.017623:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.017631:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20059 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.017912:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515816192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61575us (61802us total) trans 4294973678 rc 0/0 00000100:00100000:1.0:1789760786.017923:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6390 00000100:00100000:1.0:1789760786.017932:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764359882558 time 764421605548 state 0x2 ref 0 00000001:00080000:1.0:1789760786.017967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973678 is committed 00000100:00100000:1.0:1789760786.022118:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.022295:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816320 00000100:00100000:3.0:1789760786.022316:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816320 00000100:00100000:3.0:1789760786.022387:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764359882558 time 764421605548 state 0x0 ref 1 00000100:00100000:3.0:1789760786.022403:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764421865964 time 764421605548 state 0x1 ref 1 00000100:00100000:3.0:1789760786.022431:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764421865964 time 764487719657 state 0x0 ref 1 00000100:00100000:3.0:1789760786.022444:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.022495:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26058752, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.022505:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.022525:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760786.022530:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.022536:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.022551:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760786.024754:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760786.067269:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973679 is committed 00002000:00010000:1.0:1789760786.067347:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26058752 -> 26062848 00010000:00010000:1.0:1789760786.067362:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067375:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067386:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067394:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760786.067398:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067408:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067434:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.067443:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20060 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.067811:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45364us (45697us total) trans 4294973679 rc 0/0 00000100:00100000:1.0:1789760786.067822:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6391 00000100:00100000:1.0:1789760786.067832:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764421865964 time 764487719657 state 0x2 ref 0 00000100:00100000:3.0:1789760786.073488:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760786.073674:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816448 00000100:00100000:0.0:1789760786.073685:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816448 00000100:00100000:0.0:1789760786.073768:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764421865964 time 764487719657 state 0x0 ref 1 00000100:00100000:0.0:1789760786.073785:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764487980073 time 764487719657 state 0x1 ref 1 00000100:00100000:0.0:1789760786.073814:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764487980073 time 764539103055 state 0x0 ref 1 00000100:00100000:0.0:1789760786.073827:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760786.073878:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26062848, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:0.0:1789760786.073888:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.073905:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760786.073910:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.073917:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.073930:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.105405:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973680 is committed 00002000:00010000:2.0:1789760786.105513:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26062848 -> 26066944 00010000:00010000:2.0:1789760786.105528:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.105541:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.105550:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.105557:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.105562:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.105639:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.105669:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.105677:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20067 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.105881:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32051us (32399us total) trans 4294973680 rc 0/0 00000100:00100000:2.0:1789760786.105893:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6392 00000100:00100000:2.0:1789760786.105903:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764487980073 time 764539103055 state 0x2 ref 0 00000100:00100000:2.0:1789760786.109287:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760786.109436:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816576 00000100:00100000:1.0:1789760786.109446:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816576 00000100:00100000:1.0:1789760786.109499:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764487980073 time 764539103055 state 0x0 ref 1 00000100:00100000:1.0:1789760786.109515:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764539363471 time 764539103055 state 0x1 ref 1 00000100:00100000:1.0:1789760786.109547:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764539363471 time 764574836108 state 0x0 ref 1 00000100:00100000:1.0:1789760786.109560:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515816576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.109615:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26066944, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:1.0:1789760786.109624:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.109645:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760786.109663:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.109670:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.109685:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.135841:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26066944 -> 26071040 00010000:00010000:2.0:1789760786.135861:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.135876:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.135884:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.135891:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.135896:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.135911:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.135930:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.135938:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2006e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.136031:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515816576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26467us (26747us total) trans 4294973681 rc 0/0 00000100:00100000:2.0:1789760786.136041:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6393 00000100:00100000:2.0:1789760786.136051:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764539363471 time 764574836108 state 0x2 ref 0 00000001:00080000:0.0:1789760786.136397:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973681 is committed 00000100:00100000:1.0:1789760786.139156:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760786.139256:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816704 00000100:00100000:0.0:1789760786.139267:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816704 00000100:00100000:0.0:1789760786.139320:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764539363471 time 764574836108 state 0x0 ref 1 00000100:00100000:0.0:1789760786.139334:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764575096524 time 764574836108 state 0x1 ref 1 00000100:00100000:0.0:1789760786.139397:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764575096524 time 764604685404 state 0x0 ref 1 00000100:00100000:0.0:1789760786.139409:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760786.139455:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26071040, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:0.0:1789760786.139463:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.139478:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760786.139482:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.139489:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.139502:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.176094:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26071040 -> 26075136 00010000:00010000:3.0:1789760786.176115:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.176137:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.176147:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.176155:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760786.176160:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.176169:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.176194:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.176203:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20075 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.177359:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515816704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37943us (38204us total) trans 4294973682 rc 0/0 00000100:00100000:3.0:1789760786.177373:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6394 00000100:00100000:3.0:1789760786.177383:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764575096524 time 764604685404 state 0x2 ref 0 00000001:00080000:1.0:1789760786.178035:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973682 is committed 00000100:00100000:1.0:1789760786.181851:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.181929:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816832 00000100:00100000:3.0:1789760786.181940:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816832 00000100:00100000:3.0:1789760786.181994:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764575096524 time 764604685404 state 0x0 ref 1 00000100:00100000:3.0:1789760786.182010:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764604945820 time 764604685404 state 0x1 ref 1 00000100:00100000:3.0:1789760786.182043:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764604945820 time 764647331948 state 0x0 ref 1 00000100:00100000:3.0:1789760786.182055:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515816832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.182100:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26075136, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.182109:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.182125:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760786.182135:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.182142:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.182155:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.217849:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26075136 -> 26079232 00010000:00010000:0.0:1789760786.217869:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.217883:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.217891:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.217898:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760786.217902:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.217911:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.217929:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.217937:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d2007c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.218029:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515816832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35971us (36183us total) trans 4294973683 rc 0/0 00000100:00100000:0.0:1789760786.218048:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6395 00000100:00100000:0.0:1789760786.218057:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764604945820 time 764647331948 state 0x2 ref 0 00000100:00100000:0.0:1789760786.221334:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.221417:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515816960 00000100:00100000:3.0:1789760786.221428:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515816960 00000100:00100000:3.0:1789760786.221480:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764604945820 time 764647331948 state 0x0 ref 1 00000100:00100000:3.0:1789760786.221536:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764647592364 time 764647331948 state 0x1 ref 1 00000100:00100000:3.0:1789760786.221568:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764647592364 time 764686857345 state 0x0 ref 1 00000100:00100000:3.0:1789760786.221580:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515816960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.221623:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26079232, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.221632:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.221665:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973683 is committed 00010000:00010000:3.0:1789760786.222694:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760786.222703:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.222724:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.222742:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.261963:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26079232 -> 26083328 00010000:00010000:2.0:1789760786.261984:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.261997:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.262007:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.262020:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.262025:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.262035:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.262054:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.262061:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20083 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.262156:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515816960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40572us (40826us total) trans 4294973684 rc 0/0 00000100:00100000:2.0:1789760786.262167:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6396 00000100:00100000:2.0:1789760786.262177:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764647592364 time 764686857345 state 0x2 ref 0 00000001:00080000:2.0:1789760786.262219:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973684 is committed 00000100:00100000:1.0:1789760786.266204:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.266291:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817088 00000100:00100000:3.0:1789760786.266311:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817088 00000100:00100000:3.0:1789760786.266363:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764647592364 time 764686857345 state 0x0 ref 1 00000100:00100000:3.0:1789760786.266378:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764687117761 time 764686857345 state 0x1 ref 1 00000100:00100000:3.0:1789760786.266413:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764687117761 time 764731701778 state 0x0 ref 1 00000100:00100000:3.0:1789760786.266425:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.266475:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26083328, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.266485:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.266500:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760786.266504:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.266511:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.266524:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.293323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973685 is committed 00002000:00010000:3.0:1789760786.293549:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26083328 -> 26087424 00010000:00010000:3.0:1789760786.293564:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.293656:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.293665:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.293673:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760786.293678:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.293688:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.293727:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.293736:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2008a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.293864:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27436us (27666us total) trans 4294973685 rc 0/0 00000100:00100000:3.0:1789760786.293876:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6397 00000100:00100000:3.0:1789760786.293886:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764687117761 time 764731701778 state 0x2 ref 0 00000100:00100000:3.0:1789760786.298505:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760786.298595:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817216 00000100:00100000:0.0:1789760786.298605:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817216 00000100:00100000:0.0:1789760786.298749:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764687117761 time 764731701778 state 0x0 ref 1 00000100:00100000:0.0:1789760786.298765:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764731962194 time 764731701778 state 0x1 ref 1 00000100:00100000:0.0:1789760786.298792:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764731962194 time 764764081463 state 0x0 ref 1 00000100:00100000:0.0:1789760786.298804:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760786.298853:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26087424, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:0.0:1789760786.298862:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.298877:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760786.298882:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.298888:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.298903:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.344145:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26087424 -> 26091520 00010000:00010000:1.0:1789760786.344167:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.344182:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.344192:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.344200:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760786.344206:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.344221:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.344240:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.344248:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20091 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.344465:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515817216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45658us (45964us total) trans 4294973686 rc 0/0 00000100:00100000:1.0:1789760786.344476:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6398 00000100:00100000:1.0:1789760786.344486:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764731962194 time 764764081463 state 0x2 ref 0 00000001:00080000:3.0:1789760786.344579:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973686 is committed 00000100:00100000:3.0:1789760786.347874:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760786.348071:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817344 00000100:00100000:1.0:1789760786.348081:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817344 00000100:00100000:1.0:1789760786.348144:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764731962194 time 764764081463 state 0x0 ref 1 00000100:00100000:1.0:1789760786.348161:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764764341879 time 764764081463 state 0x1 ref 1 00000100:00100000:1.0:1789760786.348183:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764764341879 time 764813472854 state 0x0 ref 1 00000100:00100000:1.0:1789760786.348196:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.348245:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26091520, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:1.0:1789760786.348254:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.348270:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760786.348275:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.348282:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.348308:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.383097:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26091520 -> 26095616 00010000:00010000:2.0:1789760786.383117:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.383139:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.383156:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.383163:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.383168:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.383177:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.383197:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.383205:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d20098 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.383304:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515817344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35106us (35436us total) trans 4294973687 rc 0/0 00000100:00100000:2.0:1789760786.383314:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6399 00000100:00100000:2.0:1789760786.383323:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764764341879 time 764813472854 state 0x2 ref 0 00000001:00080000:3.0:1789760786.386420:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973687 is committed 00000100:00100000:2.0:1789760786.387814:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.387903:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817472 00000100:00100000:3.0:1789760786.387912:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817472 00000100:00100000:3.0:1789760786.387966:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764764341879 time 764813472854 state 0x0 ref 1 00000100:00100000:3.0:1789760786.387981:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764813733270 time 764813472854 state 0x1 ref 1 00000100:00100000:3.0:1789760786.388011:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764813733270 time 764853299916 state 0x0 ref 1 00000100:00100000:3.0:1789760786.388024:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.388071:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26095616, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.388080:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.388096:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760786.388101:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.388107:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.388120:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760786.419268:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26095616 -> 26099712 00010000:00010000:3.0:1789760786.419287:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.419319:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.419329:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.419336:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760786.419341:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.419349:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.419369:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.419378:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2009f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.419469:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515817472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31442us (31656us total) trans 4294973688 rc 0/0 00000100:00100000:3.0:1789760786.419480:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6400 00000100:00100000:3.0:1789760786.419490:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764813733270 time 764853299916 state 0x2 ref 0 00000001:00080000:0.0:1789760786.419644:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973688 is committed 00000100:00100000:3.0:1789760786.423091:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760786.423364:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817600 00000100:00100000:2.0:1789760786.423374:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817600 00000100:00100000:2.0:1789760786.423428:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764813733270 time 764853299916 state 0x0 ref 1 00000100:00100000:2.0:1789760786.423447:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764853560332 time 764853299916 state 0x1 ref 1 00000100:00100000:2.0:1789760786.423478:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764853560332 time 764888766884 state 0x0 ref 1 00000100:00100000:2.0:1789760786.423490:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760786.423536:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26099712, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:2.0:1789760786.423546:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.423561:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760786.423682:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.423690:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.423704:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.454448:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973689 is committed 00002000:00010000:1.0:1789760786.454540:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26099712 -> 26103808 00010000:00010000:1.0:1789760786.454557:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.454580:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.454589:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.454597:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760786.454602:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.454611:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.454630:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.454639:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d200a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.454756:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31264us (31671us total) trans 4294973689 rc 0/0 00000100:00100000:1.0:1789760786.454766:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6401 00000100:00100000:1.0:1789760786.454775:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764853560332 time 764888766884 state 0x2 ref 0 00000100:00100000:2.0:1789760786.458176:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.458263:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817728 00000100:00100000:3.0:1789760786.458274:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817728 00000100:00100000:3.0:1789760786.458384:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764853560332 time 764888766884 state 0x0 ref 1 00000100:00100000:3.0:1789760786.458399:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764889027300 time 764888766884 state 0x1 ref 1 00000100:00100000:3.0:1789760786.458430:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764889027300 time 764923718888 state 0x0 ref 1 00000100:00100000:3.0:1789760786.458443:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.458489:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26103808, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.458498:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.458514:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760786.458518:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.458523:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.458536:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.485995:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973690 is committed 00002000:00010000:2.0:1789760786.486103:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26103808 -> 26107904 00010000:00010000:2.0:1789760786.486120:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.486133:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.486146:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.486154:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.486158:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.486167:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.486184:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.486192:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d200ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.486296:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515817728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27842us (28116us total) trans 4294973690 rc 0/0 00000100:00100000:2.0:1789760786.486306:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6402 00000100:00100000:2.0:1789760786.486315:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764889027300 time 764923718888 state 0x2 ref 0 00000100:00100000:3.0:1789760786.489901:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760786.489996:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817856 00000100:00100000:1.0:1789760786.490006:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817856 00000100:00100000:1.0:1789760786.490068:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764889027300 time 764923718888 state 0x0 ref 1 00000100:00100000:1.0:1789760786.490083:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764923979304 time 764923718888 state 0x1 ref 1 00000100:00100000:1.0:1789760786.490114:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764923979304 time 764955403141 state 0x0 ref 1 00000100:00100000:1.0:1789760786.490126:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515817856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.490171:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26107904, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:1.0:1789760786.490181:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.490198:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760786.490202:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.490208:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.490223:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.524851:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26107904 -> 26112000 00010000:00010000:2.0:1789760786.524873:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.524893:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.524901:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.524909:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.524914:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.524921:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.524939:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.524947:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d200b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.525043:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515817856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34914us (35145us total) trans 4294973691 rc 0/0 00000100:00100000:2.0:1789760786.525052:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6403 00000100:00100000:2.0:1789760786.525062:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764923979304 time 764955403141 state 0x2 ref 0 00000001:00080000:3.0:1789760786.525152:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973691 is committed 00000100:00100000:3.0:1789760786.529225:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760786.529335:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515817984 00000100:00100000:1.0:1789760786.529345:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515817984 00000100:00100000:1.0:1789760786.529403:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764923979304 time 764955403141 state 0x0 ref 1 00000100:00100000:1.0:1789760786.529419:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764955663557 time 764955403141 state 0x1 ref 1 00000100:00100000:1.0:1789760786.529454:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764955663557 time 764994743084 state 0x0 ref 1 00000100:00100000:1.0:1789760786.529465:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515817984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.529513:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26112000, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:1.0:1789760786.529521:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.529537:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760786.529541:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.529547:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.529560:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.561788:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973692 is committed 00002000:00010000:0.0:1789760786.561887:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26112000 -> 26116096 00010000:00010000:0.0:1789760786.561903:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.561916:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.561924:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.561931:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760786.561935:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.561944:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.561964:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.561971:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d200bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.562055:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515817984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32586us (32834us total) trans 4294973692 rc 0/0 00000100:00100000:0.0:1789760786.562065:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6404 00000100:00100000:0.0:1789760786.562075:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764955663557 time 764994743084 state 0x2 ref 0 00000100:00100000:2.0:1789760786.566940:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760786.567016:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818112 00000100:00100000:0.0:1789760786.567025:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818112 00000100:00100000:0.0:1789760786.567078:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764955663557 time 764994743084 state 0x0 ref 1 00000100:00100000:0.0:1789760786.567092:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764995003500 time 764994743084 state 0x1 ref 1 00000100:00100000:0.0:1789760786.567122:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764995003500 time 765032410873 state 0x0 ref 1 00000100:00100000:0.0:1789760786.567134:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515818112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760786.567180:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26116096, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:0.0:1789760786.567188:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.567203:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760786.567208:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.567214:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.567229:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.595025:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26116096 -> 26120192 00010000:00010000:2.0:1789760786.595045:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.595060:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.595070:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.595077:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.595082:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.595091:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.595111:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.595118:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d200c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.595212:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515818112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28074us (28276us total) trans 4294973693 rc 0/0 00000100:00100000:2.0:1789760786.595223:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6405 00000100:00100000:2.0:1789760786.595233:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764995003500 time 765032410873 state 0x2 ref 0 00000001:00080000:1.0:1789760786.597782:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973693 is committed 00000100:00100000:2.0:1789760786.599993:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760786.600067:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818240 00000100:00100000:3.0:1789760786.600077:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818240 00000100:00100000:3.0:1789760786.600131:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 764995003500 time 765032410873 state 0x0 ref 1 00000100:00100000:3.0:1789760786.600149:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765032671289 time 765032410873 state 0x1 ref 1 00000100:00100000:3.0:1789760786.600178:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765032671289 time 765065467694 state 0x0 ref 1 00000100:00100000:3.0:1789760786.600191:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515818240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760786.600237:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26120192, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:3.0:1789760786.600246:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.600264:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.600269:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.600274:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.600285:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.625598:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26120192 -> 26124288 00010000:00010000:1.0:1789760786.625619:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625632:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625642:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625659:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760786.625664:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625674:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625693:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.625701:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d200c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.625821:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515818240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25626us (25831us total) trans 4294973694 rc 0/0 00000100:00100000:1.0:1789760786.625832:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6406 00000100:00100000:1.0:1789760786.625842:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765032671289 time 765065467694 state 0x2 ref 0 00000001:00080000:2.0:1789760786.627023:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973694 is committed 00000100:00100000:0.0:1789760786.629367:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760786.629455:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818368 00000100:00100000:2.0:1789760786.629476:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818368 00000100:00100000:2.0:1789760786.629530:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765032671289 time 765065467694 state 0x0 ref 1 00000100:00100000:2.0:1789760786.629545:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765065728110 time 765065467694 state 0x1 ref 1 00000100:00100000:2.0:1789760786.629584:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765065728110 time 765094873244 state 0x0 ref 1 00000100:00100000:2.0:1789760786.629597:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515818368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760786.629659:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26124288, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:2.0:1789760786.629669:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.629685:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.629690:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.629696:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.629745:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.681183:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26124288 -> 26128384 00010000:00010000:2.0:1789760786.681200:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.681210:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.681215:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.681220:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.681223:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.681230:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.681244:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.681249:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.681324:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515818368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51724us (51961us total) trans 4294973695 rc 0/0 00000100:00100000:2.0:1789760786.681334:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6407 00000100:00100000:2.0:1789760786.681343:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765065728110 time 765094873244 state 0x2 ref 0 00000001:00080000:0.0:1789760786.681415:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973695 is committed 00000100:00100000:0.0:1789760786.701360:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760786.701484:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818496 00000100:00100000:2.0:1789760786.701495:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818496 00000100:00100000:2.0:1789760786.701550:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765065728110 time 765094873244 state 0x0 ref 1 00000100:00100000:2.0:1789760786.701566:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765095133660 time 765094873244 state 0x1 ref 1 00000100:00100000:2.0:1789760786.701628:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765095133660 time 765166916919 state 0x0 ref 1 00000100:00100000:2.0:1789760786.701641:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515818496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760786.701701:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26128384, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:2.0:1789760786.701730:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.701746:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.701751:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.701757:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.701771:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.739754:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973696 is committed 00002000:00010000:2.0:1789760786.739890:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26128384 -> 26132480 00010000:00010000:2.0:1789760786.739905:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739927:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739942:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739950:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.739955:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739964:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739983:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739991:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.740079:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515818496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38435us (38727us total) trans 4294973696 rc 0/0 00000100:00100000:2.0:1789760786.740295:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6408 00000100:00100000:2.0:1789760786.740305:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765095133660 time 765166916919 state 0x2 ref 0 00000100:00100000:3.0:1789760786.746126:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760786.746184:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818624 00000100:00100000:2.0:1789760786.746194:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818624 00000100:00100000:2.0:1789760786.746245:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765095133660 time 765166916919 state 0x0 ref 1 00000100:00100000:2.0:1789760786.746261:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765167177335 time 765166916919 state 0x1 ref 1 00000100:00100000:2.0:1789760786.746295:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765167177335 time 765211584212 state 0x0 ref 1 00000100:00100000:2.0:1789760786.746308:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515818624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760786.746357:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26132480, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:2.0:1789760786.746366:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746381:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.746386:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.746402:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746416:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.760007:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760786.816631:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973697 is committed 00002000:00010000:2.0:1789760786.816939:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26132480 -> 26136576 00010000:00010000:2.0:1789760786.816955:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.816969:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.816979:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.816992:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.816997:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.817006:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.817024:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.817031:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.817132:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515818624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70813us (71003us total) trans 4294973697 rc 0/0 00000100:00100000:2.0:1789760786.817143:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6409 00000100:00100000:2.0:1789760786.817153:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765167177335 time 765211584212 state 0x2 ref 0 00000100:00100000:0.0:1789760786.821657:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760786.821850:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818752 00000100:00100000:1.0:1789760786.821860:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818752 00000100:00100000:1.0:1789760786.821915:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765167177335 time 765211584212 state 0x0 ref 1 00000100:00100000:1.0:1789760786.821932:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765211844628 time 765211584212 state 0x1 ref 1 00000100:00100000:1.0:1789760786.821964:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765211844628 time 765287253345 state 0x0 ref 1 00000100:00100000:1.0:1789760786.821977:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515818752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.822028:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26136576, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc807 00010000:00010000:1.0:1789760786.822037:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.822053:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.822057:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.822064:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.822078:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.870018:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973698 is committed 00002000:00010000:0.0:1789760786.870104:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26136576 -> 26140672 00002000:00010000:0.0:1789760786.870109:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:0.0:1789760786.870112:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00010000:00010000:0.0:1789760786.870184:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.870195:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.870201:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.870206:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760786.870210:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.870262:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.870280:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.870286:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d200e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.870460:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515818752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48481us (48818us total) trans 4294973698 rc 0/0 00000100:00100000:0.0:1789760786.870468:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6410 00000100:00100000:0.0:1789760786.870482:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765211844628 time 765287253345 state 0x2 ref 0 00000100:00100000:3.0:1789760786.874343:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760786.875426:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515818880 00000100:00100000:0.0:1789760786.875439:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515818880 00000100:00100000:1.0:1789760786.875780:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765211844628 time 765287253345 state 0x0 ref 1 00000100:00100000:1.0:1789760786.875796:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765287513761 time 765287253345 state 0x1 ref 1 00000100:00100000:1.0:1789760786.875830:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765287513761 time 765341119676 state 0x0 ref 1 00000100:00100000:1.0:1789760786.875844:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515818880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.875894:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26140672, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760786.875904:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.875921:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.875925:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.875931:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.875944:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.920980:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973699 is committed 00002000:00010000:2.0:1789760786.922155:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26140672 -> 26144768 00010000:00010000:2.0:1789760786.922184:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.922196:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.922210:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.922217:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760786.922222:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.922231:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.922366:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.922376:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d200ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.922630:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515818880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46728us (48235us total) trans 4294973699 rc 0/0 00000100:00100000:2.0:1789760786.922649:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6411 00000100:00100000:2.0:1789760786.922660:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765287513761 time 765341119676 state 0x2 ref 0 00000100:00100000:3.0:1789760786.934152:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760786.934696:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819008 00000100:00100000:1.0:1789760786.934745:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819008 00000100:00100000:1.0:1789760786.934798:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765287513761 time 765341119676 state 0x0 ref 1 00000100:00100000:1.0:1789760786.934811:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765341380092 time 765341119676 state 0x1 ref 1 00000100:00100000:1.0:1789760786.934840:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765341380092 time 765400129731 state 0x0 ref 1 00000100:00100000:1.0:1789760786.934853:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515819008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760786.934898:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26144768, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760786.934907:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.934923:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.934928:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.934934:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.934947:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.990511:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26144768 -> 26148864 00010000:00010000:1.0:1789760786.990541:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.990554:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.990826:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973700 is committed 00010000:00010000:1.0:1789760786.990976:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.990985:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760786.990989:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.990998:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.991015:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.991023:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d200f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.991145:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515819008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56290us (56997us total) trans 4294973700 rc 0/0 00000100:00100000:1.0:1789760786.991155:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6412 00000100:00100000:1.0:1789760786.991163:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765341380092 time 765400129731 state 0x2 ref 0 00000100:00100000:1.0:1789760786.996052:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760786.996143:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819136 00000100:00100000:2.0:1789760786.996153:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819136 00000100:00100000:2.0:1789760786.996207:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765341380092 time 765400129731 state 0x0 ref 1 00000100:00100000:2.0:1789760786.996358:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765400390147 time 765400129731 state 0x1 ref 1 00000100:00100000:2.0:1789760786.996496:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765400390147 time 765461783672 state 0x0 ref 1 00000100:00100000:2.0:1789760786.996510:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515819136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760786.996791:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26148864, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760786.996802:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.996818:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.996822:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.996828:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.996842:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.047621:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973701 is committed 00002000:00010000:3.0:1789760787.051419:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26148864 -> 26152960 00010000:00010000:3.0:1789760787.051501:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051520:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051529:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051536:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760787.051541:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051703:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051754:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.051763:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d200fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.051904:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515819136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55391us (55856us total) trans 4294973701 rc 0/0 00000100:00100000:3.0:1789760787.051914:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6413 00000100:00100000:3.0:1789760787.051924:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765400390147 time 765461783672 state 0x2 ref 0 00000100:00100000:3.0:1789760787.056371:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.056500:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819264 00000100:00100000:2.0:1789760787.056511:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819264 00000100:00100000:2.0:1789760787.056574:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765400390147 time 765461783672 state 0x0 ref 1 00000100:00100000:2.0:1789760787.056590:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765462044088 time 765461783672 state 0x1 ref 1 00000100:00100000:2.0:1789760787.056623:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765462044088 time 765521912309 state 0x0 ref 1 00000100:00100000:2.0:1789760787.056636:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515819264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.056694:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26152960, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760787.056704:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.063806:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760787.063811:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.063819:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.063836:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.124789:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973702 is committed 00002000:00010000:3.0:1789760787.124858:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26152960 -> 26157056 00010000:00010000:3.0:1789760787.124874:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.124888:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.124897:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.124904:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760787.124910:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.124919:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.124945:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.124953:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20101 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.125028:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515819264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 68390us (68662us total) trans 4294973702 rc 0/0 00000100:00100000:3.0:1789760787.125037:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6414 00000100:00100000:3.0:1789760787.125047:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765462044088 time 765521912309 state 0x2 ref 0 00000100:00100000:2.0:1789760787.135996:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760787.136070:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819392 00000100:00100000:1.0:1789760787.136090:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819392 00000100:00100000:1.0:1789760787.136143:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765462044088 time 765521912309 state 0x0 ref 1 00000100:00100000:1.0:1789760787.136158:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765522172725 time 765521912309 state 0x1 ref 1 00000100:00100000:1.0:1789760787.136192:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765522172725 time 765601481118 state 0x0 ref 1 00000100:00100000:1.0:1789760787.136214:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515819392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760787.136272:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26157056, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760787.136281:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.136297:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760787.136301:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.136308:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.136322:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.182115:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973703 is committed 00002000:00010000:2.0:1789760787.182188:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26157056 -> 26161152 00010000:00010000:2.0:1789760787.182203:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.182215:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.182223:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.182230:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760787.182234:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.182247:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.182264:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.182270:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20108 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.182394:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515819392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46177us (46403us total) trans 4294973703 rc 0/0 00000100:00100000:2.0:1789760787.182405:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6415 00000100:00100000:2.0:1789760787.182415:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765522172725 time 765601481118 state 0x2 ref 0 00000100:00100000:0.0:1789760787.185863:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760787.185957:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819520 00000100:00100000:1.0:1789760787.185966:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819520 00000100:00100000:1.0:1789760787.186014:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765522172725 time 765601481118 state 0x0 ref 1 00000100:00100000:1.0:1789760787.186028:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765601741534 time 765601481118 state 0x1 ref 1 00000100:00100000:1.0:1789760787.186221:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765601741534 time 765651508658 state 0x0 ref 1 00000100:00100000:1.0:1789760787.186233:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515819520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760787.186274:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26161152, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760787.186684:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.186701:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760787.186803:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.186812:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.186828:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.241111:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973704 is committed 00002000:00010000:3.0:1789760787.241302:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26161152 -> 26165248 00010000:00010000:3.0:1789760787.241319:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.241332:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.241342:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.241349:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760787.241354:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.241372:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.241393:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.241401:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2010f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.241499:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515819520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55262us (55639us total) trans 4294973704 rc 0/0 00000100:00100000:3.0:1789760787.241513:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6416 00000100:00100000:3.0:1789760787.241523:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765601741534 time 765651508658 state 0x2 ref 0 00000100:00100000:3.0:1789760787.245541:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760787.246034:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819648 00000100:00100000:1.0:1789760787.246044:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819648 00000100:00100000:1.0:1789760787.246097:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765601741534 time 765651508658 state 0x0 ref 1 00000100:00100000:1.0:1789760787.246112:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765651769074 time 765651508658 state 0x1 ref 1 00000100:00100000:1.0:1789760787.246142:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765651769074 time 765711431020 state 0x0 ref 1 00000100:00100000:1.0:1789760787.246154:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515819648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760787.246204:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26165248, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760787.246212:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.246229:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760787.246233:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.246239:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.246251:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.296405:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973705 is committed 00002000:00010000:3.0:1789760787.296474:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26165248 -> 26169344 00010000:00010000:3.0:1789760787.296490:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.296501:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.296521:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.296528:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760787.296532:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.296540:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.296559:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.296566:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20116 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.296671:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515819648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50514us (51136us total) trans 4294973705 rc 0/0 00000100:00100000:3.0:1789760787.296689:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6417 00000100:00100000:3.0:1789760787.296698:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765651769074 time 765711431020 state 0x2 ref 0 00000100:00100000:0.0:1789760787.305418:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760787.305598:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819776 00000100:00100000:3.0:1789760787.305608:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819776 00000100:00100000:3.0:1789760787.305673:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765651769074 time 765711431020 state 0x0 ref 1 00000100:00100000:3.0:1789760787.305695:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765711691436 time 765711431020 state 0x1 ref 1 00000100:00100000:3.0:1789760787.305763:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765711691436 time 765771051415 state 0x0 ref 1 00000100:00100000:3.0:1789760787.305777:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515819776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760787.305823:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26169344, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:3.0:1789760787.305831:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.305845:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760787.305849:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.305855:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.305868:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.349183:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973706 is committed 00002000:00010000:0.0:1789760787.349327:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26169344 -> 26173440 00010000:00010000:0.0:1789760787.349343:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.349356:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.349386:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.349395:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760787.349400:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.349410:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.349431:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.349440:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2011d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.349544:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515819776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43764us (44127us total) trans 4294973706 rc 0/0 00000100:00100000:0.0:1789760787.349556:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6418 00000100:00100000:0.0:1789760787.349803:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765711691436 time 765771051415 state 0x2 ref 0 00000100:00100000:0.0:1789760787.352682:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760787.352773:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515819904 00000100:00100000:3.0:1789760787.352782:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515819904 00000100:00100000:3.0:1789760787.352836:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765711691436 time 765771051415 state 0x0 ref 1 00000100:00100000:3.0:1789760787.352852:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765771311831 time 765771051415 state 0x1 ref 1 00000100:00100000:3.0:1789760787.352876:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765771311831 time 765818165472 state 0x0 ref 1 00000100:00100000:3.0:1789760787.352887:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515819904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760787.352933:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26173440, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:3.0:1789760787.352941:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.352955:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760787.352960:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.352966:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.352980:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.401659:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973707 is committed 00002000:00010000:0.0:1789760787.401743:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26173440 -> 26177536 00010000:00010000:0.0:1789760787.401761:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.401774:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.401783:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.401791:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760787.401796:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.401810:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.401832:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.401840:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20124 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.401928:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515819904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49037us (49249us total) trans 4294973707 rc 0/0 00000100:00100000:0.0:1789760787.401939:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6419 00000100:00100000:0.0:1789760787.401949:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765771311831 time 765818165472 state 0x2 ref 0 00000100:00100000:3.0:1789760787.413210:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.413424:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820032 00000100:00100000:2.0:1789760787.413434:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820032 00000100:00100000:2.0:1789760787.413651:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765771311831 time 765818165472 state 0x0 ref 1 00000100:00100000:2.0:1789760787.413667:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765818425888 time 765818165472 state 0x1 ref 1 00000100:00100000:2.0:1789760787.413700:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765818425888 time 765878988288 state 0x0 ref 1 00000100:00100000:2.0:1789760787.413746:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515820032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.413794:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26177536, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760787.413804:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.413822:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760787.413826:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.413832:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.413845:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.470500:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973708 is committed 00002000:00010000:3.0:1789760787.471273:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26177536 -> 26181632 00010000:00010000:3.0:1789760787.471289:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471375:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471384:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471391:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760787.471396:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471404:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471426:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.471434:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2012b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.471634:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515820032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57887us (58428us total) trans 4294973708 rc 0/0 00000100:00100000:3.0:1789760787.471807:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6420 00000100:00100000:3.0:1789760787.471818:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765818425888 time 765878988288 state 0x2 ref 0 00000100:00100000:3.0:1789760787.475878:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.475947:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820160 00000100:00100000:2.0:1789760787.475957:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820160 00000100:00100000:2.0:1789760787.476013:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765818425888 time 765878988288 state 0x0 ref 1 00000100:00100000:2.0:1789760787.476029:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765879248704 time 765878988288 state 0x1 ref 1 00000100:00100000:2.0:1789760787.476062:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765879248704 time 765941351062 state 0x0 ref 1 00000100:00100000:2.0:1789760787.476075:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515820160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.476120:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26181632, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760787.476128:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.476144:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760787.476149:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.476155:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.476174:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.518769:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973709 is committed 00002000:00010000:2.0:1789760787.519247:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26181632 -> 26185728 00010000:00010000:2.0:1789760787.519264:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.519277:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.519287:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.519295:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760787.519300:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.519310:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.519329:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.519338:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20132 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.519606:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515820160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43528us (43730us total) trans 4294973709 rc 0/0 00000100:00100000:2.0:1789760787.519617:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6421 00000100:00100000:2.0:1789760787.519629:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765879248704 time 765941351062 state 0x2 ref 0 00000100:00100000:0.0:1789760787.527032:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760787.527256:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820288 00000100:00100000:1.0:1789760787.527268:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820288 00000100:00100000:1.0:1789760787.527323:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765879248704 time 765941351062 state 0x0 ref 1 00000100:00100000:1.0:1789760787.527342:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765941611478 time 765941351062 state 0x1 ref 1 00000100:00100000:1.0:1789760787.527405:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765941611478 time 765992693443 state 0x0 ref 1 00000100:00100000:1.0:1789760787.527418:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515820288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760787.527468:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26185728, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760787.527477:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527493:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760787.527497:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.527503:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.527516:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760787.567594:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26185728 -> 26189824 00010000:00010000:2.0:1789760787.567613:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.567627:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.567636:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.567653:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760787.567658:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.567668:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.567692:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.567704:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20139 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.567819:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515820288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40398us (40794us total) trans 4294973710 rc 0/0 00000100:00100000:2.0:1789760787.567829:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6422 00000100:00100000:2.0:1789760787.567839:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765941611478 time 765992693443 state 0x2 ref 0 00000001:00080000:0.0:1789760787.568279:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973710 is committed 00000100:00100000:1.0:1789760787.574852:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760787.574937:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820416 00000100:00100000:3.0:1789760787.574946:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820416 00000100:00100000:3.0:1789760787.574998:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765941611478 time 765992693443 state 0x0 ref 1 00000100:00100000:3.0:1789760787.575015:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765992953859 time 765992693443 state 0x1 ref 1 00000100:00100000:3.0:1789760787.575091:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765992953859 time 766040379245 state 0x0 ref 1 00000100:00100000:3.0:1789760787.575104:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515820416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760787.575151:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26189824, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:3.0:1789760787.575160:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.575175:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760787.575180:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.575186:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.575201:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.617485:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26189824 -> 26193920 00010000:00010000:1.0:1789760787.617505:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.617517:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.617525:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.617533:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760787.617537:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.617545:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.617573:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.617581:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20140 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.617681:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515820416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42574us (42834us total) trans 4294973711 rc 0/0 00000100:00100000:1.0:1789760787.617692:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6423 00000100:00100000:1.0:1789760787.617702:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765992953859 time 766040379245 state 0x2 ref 0 00000001:00080000:0.0:1789760787.618199:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973711 is committed 00000100:00100000:1.0:1789760787.625194:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760787.628190:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820544 00000100:00100000:0.0:1789760787.628203:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820544 00000100:00100000:0.0:1789760787.628262:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 765992953859 time 766040379245 state 0x0 ref 1 00000100:00100000:0.0:1789760787.628276:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766040639661 time 766040379245 state 0x1 ref 1 00000100:00100000:0.0:1789760787.628309:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766040639661 time 766093598430 state 0x0 ref 1 00000100:00100000:0.0:1789760787.628320:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515820544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760787.628365:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26193920, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:0.0:1789760787.628373:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.628454:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.628458:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.628464:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.628477:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.664851:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973712 is committed 00002000:00010000:2.0:1789760787.665078:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26193920 -> 26198016 00010000:00010000:2.0:1789760787.665093:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.665105:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.665114:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.665121:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760787.665125:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.665134:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.665159:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.665167:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20147 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.665258:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515820544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36935us (40068us total) trans 4294973712 rc 0/0 00000100:00100000:2.0:1789760787.665269:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6424 00000100:00100000:2.0:1789760787.665279:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766040639661 time 766093598430 state 0x2 ref 0 00000100:00100000:2.0:1789760787.669076:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760787.669169:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820672 00000100:00100000:0.0:1789760787.669179:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820672 00000100:00100000:0.0:1789760787.669229:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766040639661 time 766093598430 state 0x0 ref 1 00000100:00100000:0.0:1789760787.669242:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766093858846 time 766093598430 state 0x1 ref 1 00000100:00100000:0.0:1789760787.669271:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766093858846 time 766134560039 state 0x0 ref 1 00000100:00100000:0.0:1789760787.669283:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515820672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760787.669330:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26198016, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:0.0:1789760787.669339:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.669358:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.669363:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.669370:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.669384:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.699389:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973713 is committed 00002000:00010000:0.0:1789760787.699583:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26198016 -> 26202112 00010000:00010000:0.0:1789760787.699598:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.699611:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.699619:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.699626:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760787.699631:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.699731:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.699751:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.699760:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d2014e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.699985:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515820672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30699us (30913us total) trans 4294973713 rc 0/0 00000100:00100000:0.0:1789760787.699996:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6425 00000100:00100000:0.0:1789760787.700005:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766093858846 time 766134560039 state 0x2 ref 0 00000100:00100000:3.0:1789760787.708495:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.708828:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820800 00000100:00100000:2.0:1789760787.708838:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820800 00000100:00100000:2.0:1789760787.708891:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766093858846 time 766134560039 state 0x0 ref 1 00000100:00100000:2.0:1789760787.708905:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766134820455 time 766134560039 state 0x1 ref 1 00000100:00100000:2.0:1789760787.709038:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766134820455 time 766174325716 state 0x0 ref 1 00000100:00100000:2.0:1789760787.709051:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515820800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.709098:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26202112, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760787.709107:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.709128:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.709133:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.709139:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.709173:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.736247:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973714 is committed 00002000:00010000:1.0:1789760787.736444:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26202112 -> 26206208 00010000:00010000:1.0:1789760787.736460:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.736473:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.736483:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.736491:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760787.736496:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.736507:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.736527:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.736535:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20155 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.736960:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515820800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27905us (28468us total) trans 4294973714 rc 0/0 00000100:00100000:1.0:1789760787.736972:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6426 00000100:00100000:1.0:1789760787.736981:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766134820455 time 766174325716 state 0x2 ref 0 00000100:00100000:1.0:1789760787.742660:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.742828:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515820928 00000100:00100000:2.0:1789760787.742838:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515820928 00000100:00100000:2.0:1789760787.742891:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766134820455 time 766174325716 state 0x0 ref 1 00000100:00100000:2.0:1789760787.742907:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766174586132 time 766174325716 state 0x1 ref 1 00000100:00100000:2.0:1789760787.742943:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766174586132 time 766208231642 state 0x0 ref 1 00000100:00100000:2.0:1789760787.742956:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515820928:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.743004:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26206208, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760787.743014:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.743029:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.743033:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.743040:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.743055:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760787.769603:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26206208 -> 26210304 00010000:00010000:0.0:1789760787.769625:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.769733:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973715 is committed 00010000:00010000:0.0:1789760787.769853:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.769863:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.769871:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760787.769876:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.769886:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.769910:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.769918:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2015c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.770007:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515820928:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27048us (27367us total) trans 4294973715 rc 0/0 00000100:00100000:0.0:1789760787.770018:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6427 00000100:00100000:0.0:1789760787.770026:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766174586132 time 766208231642 state 0x2 ref 0 00010000:00010000:3.0:1789760787.774555:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760787.774688:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760787.775194:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821056 00000100:00100000:1.0:1789760787.775204:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821056 00000100:00100000:1.0:1789760787.775256:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766174586132 time 766208231642 state 0x0 ref 1 00000100:00100000:1.0:1789760787.775272:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766208492058 time 766208231642 state 0x1 ref 1 00000100:00100000:1.0:1789760787.775378:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766208492058 time 766240666248 state 0x0 ref 1 00000100:00100000:1.0:1789760787.775393:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515821056:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760787.775443:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26210304, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:1.0:1789760787.775452:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.775468:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.775473:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.775480:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.775494:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.804462:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973716 is committed 00002000:00010000:1.0:1789760787.804524:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26210304 -> 26214400 00010000:00010000:1.0:1789760787.804539:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.804552:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.804631:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.804648:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760787.804653:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.804662:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.804682:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.804691:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20163 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.804807:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515821056:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29412us (30120us total) trans 4294973716 rc 0/0 00000100:00100000:1.0:1789760787.804817:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6428 00000100:00100000:1.0:1789760787.804826:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766208492058 time 766240666248 state 0x2 ref 0 00000100:00100000:1.0:1789760787.812506:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760787.814914:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821184 00000100:00100000:3.0:1789760787.814930:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821184 00000100:00100000:2.0:1789760787.815435:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766208492058 time 766240666248 state 0x0 ref 1 00000100:00100000:2.0:1789760787.815459:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766240926664 time 766240666248 state 0x1 ref 1 00000100:00100000:2.0:1789760787.815546:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766240926664 time 766280834663 state 0x0 ref 1 00000100:00100000:2.0:1789760787.815569:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515821184:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.815617:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26214400, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc807 00010000:00010000:2.0:1789760787.815626:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.815649:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.815654:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.815660:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.815673:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.868059:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973717 is committed 00002000:00010000:1.0:1789760787.868293:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26214400 -> 26218496 00002000:00010000:1.0:1789760787.868298:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:1.0:1789760787.868300:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00002000:00010000:1.0:1789760787.868303:0:6712:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:1.0:1789760787.868316:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.868328:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.868338:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.868345:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760787.868350:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.868360:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.868380:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.868388:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2016a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.868482:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515821184:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52920us (55981us total) trans 4294973717 rc 0/0 00000100:00100000:1.0:1789760787.868493:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6429 00000100:00100000:1.0:1789760787.868502:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766240926664 time 766280834663 state 0x2 ref 0 00000100:00100000:0.0:1789760787.880974:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.881090:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821312 00000100:00100000:2.0:1789760787.881100:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821312 00000100:00100000:2.0:1789760787.881152:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766240926664 time 766280834663 state 0x0 ref 1 00000100:00100000:2.0:1789760787.881167:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766281095079 time 766280834663 state 0x1 ref 1 00000100:00100000:2.0:1789760787.881229:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766281095079 time 766346518455 state 0x0 ref 1 00000100:00100000:2.0:1789760787.881242:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515821312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760787.881288:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26218496, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:2.0:1789760787.881297:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.881312:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.881316:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.881322:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.881335:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.921922:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973718 is committed 00002000:00010000:1.0:1789760787.922071:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26218496 -> 26222592 00010000:00010000:1.0:1789760787.922086:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.922099:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.922107:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.922114:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760787.922119:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.922129:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.922146:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.922154:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20171 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.922241:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515821312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40997us (41272us total) trans 4294973718 rc 0/0 00000100:00100000:1.0:1789760787.922251:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6430 00000100:00100000:1.0:1789760787.922261:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766281095079 time 766346518455 state 0x2 ref 0 00000100:00100000:0.0:1789760787.939825:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760787.940694:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821440 00000100:00100000:2.0:1789760787.940735:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821440 00000100:00100000:2.0:1789760787.940792:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766281095079 time 766346518455 state 0x0 ref 1 00000100:00100000:2.0:1789760787.940807:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766346778871 time 766346518455 state 0x1 ref 1 00000100:00100000:1.0:1789760787.940912:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766346778871 time 766406199206 state 0x0 ref 1 00000100:00100000:1.0:1789760787.940928:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760787.940979:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26222592, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760787.940987:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.941001:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.941005:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.941011:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.941023:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.965932:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544421504:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966020:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.966052:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93b10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544421248:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966086:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760787.966094:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93c760700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544421120:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966125:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.966135:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760787.966154:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544421632:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.966166:0:3320:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544421760:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966186:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.966200:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760787.966220:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae93b109880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544421376:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.966245:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421248 00000100:00100000:2.0:1789760787.966251:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.966255:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421248 00000100:00100000:2.0:1789760787.966288:0:8226:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421504 00000100:00100000:2.0:1789760787.966293:0:8226:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421504 00000100:00100000:3.0:1789760787.966307:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421632 00000100:00100000:3.0:1789760787.966312:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421632 00000100:00100000:3.0:1789760787.966337:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421376 00000100:00100000:3.0:1789760787.966340:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421376 00000100:00100000:2.0:1789760787.966361:0:8226:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421760 00000100:00100000:3.0:1789760787.966364:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:2.0:1789760787.966367:0:8226:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421760 00000100:00100000:3.0:1789760787.966376:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544421248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966399:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789760787.966408:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544421504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.966433:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1876699544421248:12345-0@lo:400:kworker.0 Request processed in 58us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.966442:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:2.0:1789760787.966459:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544421504:12345-0@lo:400:kworker.0 Request processed in 51us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.966464:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:2.0:1789760787.966473:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:3.0:1789760787.966478:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544421632:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966493:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789760787.966502:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544421760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.966513:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544421632:12345-0@lo:400:kworker.0 Request processed in 39us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.966521:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:3.0:1789760787.966539:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:2.0:1789760787.966544:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3320:x1876699544421760:12345-0@lo:400:kworker.0 Request processed in 41us (342us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.966552:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544421376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966556:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789760787.966577:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421120 00000100:00100000:2.0:1789760787.966581:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421120 00000100:00100000:2.0:1789760787.966604:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789760787.966610:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544421120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966655:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544421120:12345-0@lo:400:kworker.0 Request processed in 44us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.966664:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00080000:2.0:1789760787.966696:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760787.966730:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760787.966733:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760787.966759:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93c760700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544421120:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966772:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544421632:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966782:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544421504:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966847:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93b10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3320:1876699544421248:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.966857:0:3320:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae8057b8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3320:1876699544421760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.967165:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544421376:12345-0@lo:400:kworker.0 Request processed in 616us (914us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.967173:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:2.0:1789760787.967413:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae93b109880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544421376:0@lo:400:kworker.0 00000001:00080000:1.0:1789760787.995349:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973719 is committed 00002000:00010000:3.0:1789760787.997163:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26222592 -> 26226688 00010000:00010000:3.0:1789760787.997181:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.997196:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.997206:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.997214:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760787.997219:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.997229:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.997249:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.997258:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20178 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.997365:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56436us (57545us total) trans 4294973719 rc 0/0 00000100:00100000:3.0:1789760787.997376:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6431 00000100:00100000:3.0:1789760787.997385:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766346778871 time 766406199206 state 0x2 ref 0 00000100:00100000:3.0:1789760788.002400:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.006236:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821568 00000100:00100000:1.0:1789760788.006249:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821568 00000100:00100000:0.0:1789760788.006476:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766346778871 time 766406199206 state 0x0 ref 1 00000100:00100000:0.0:1789760788.006491:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766406459622 time 766406199206 state 0x1 ref 1 00000100:00100000:0.0:1789760788.006521:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766406459622 time 766471810182 state 0x0 ref 1 00000100:00100000:0.0:1789760788.006533:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760788.006576:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26226688, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:0.0:1789760788.006585:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.006600:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760788.006605:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.006610:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.006624:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760788.015182:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760788.052174:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973720 is committed 00002000:00010000:1.0:1789760788.052259:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26226688 -> 26230784 00010000:00010000:1.0:1789760788.052277:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.052289:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.052302:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.052309:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.052314:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.052323:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.052343:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.052350:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d2017f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.052483:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45948us (50087us total) trans 4294973720 rc 0/0 00000100:00100000:1.0:1789760788.052493:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6432 00000100:00100000:1.0:1789760788.052503:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766406459622 time 766471810182 state 0x2 ref 0 00000100:00100000:0.0:1789760788.063080:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.063190:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821696 00000100:00100000:2.0:1789760788.063201:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821696 00000100:00100000:2.0:1789760788.063251:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766406459622 time 766471810182 state 0x0 ref 1 00000100:00100000:2.0:1789760788.063266:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766472070598 time 766471810182 state 0x1 ref 1 00000100:00100000:2.0:1789760788.063286:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766472070598 time 766528575943 state 0x0 ref 1 00000100:00100000:2.0:1789760788.063297:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760788.063340:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26230784, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:2.0:1789760788.063349:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.063369:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760788.063373:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.063379:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.063391:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.098256:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973721 is committed 00002000:00010000:2.0:1789760788.098368:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26230784 -> 26234880 00010000:00010000:2.0:1789760788.098384:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.098397:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.098406:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.098414:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760788.098419:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.098428:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.098448:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.098457:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20186 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.098653:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35351us (35577us total) trans 4294973721 rc 0/0 00000100:00100000:2.0:1789760788.098665:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6433 00000100:00100000:2.0:1789760788.098770:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766472070598 time 766528575943 state 0x2 ref 0 00000100:00100000:0.0:1789760788.103866:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.103949:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821824 00000100:00100000:1.0:1789760788.103959:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821824 00000100:00100000:1.0:1789760788.104013:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766472070598 time 766528575943 state 0x0 ref 1 00000100:00100000:1.0:1789760788.104032:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766528836359 time 766528575943 state 0x1 ref 1 00000100:00100000:1.0:1789760788.104072:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766528836359 time 766569360748 state 0x0 ref 1 00000100:00100000:1.0:1789760788.104084:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760788.104133:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26234880, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760788.104143:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.104171:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760788.104177:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.104183:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.104198:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.135519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973722 is committed 00002000:00010000:1.0:1789760788.135933:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26234880 -> 26238976 00010000:00010000:1.0:1789760788.135950:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.135962:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.135971:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.135983:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.135988:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.135998:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.136017:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.136025:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2018d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.136331:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515821824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32244us (32470us total) trans 4294973722 rc 0/0 00000100:00100000:1.0:1789760788.136343:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6434 00000100:00100000:1.0:1789760788.136352:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766528836359 time 766569360748 state 0x2 ref 0 00000100:00100000:1.0:1789760788.142395:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.142499:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515821952 00000100:00100000:2.0:1789760788.142510:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515821952 00000100:00100000:2.0:1789760788.142627:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766528836359 time 766569360748 state 0x0 ref 1 00000100:00100000:2.0:1789760788.142653:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766569621164 time 766569360748 state 0x1 ref 1 00000100:00100000:2.0:1789760788.142789:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766569621164 time 766608076067 state 0x0 ref 1 00000100:00100000:2.0:1789760788.142802:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515821952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760788.142851:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26238976, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:2.0:1789760788.142860:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.142877:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760788.142881:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.142887:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.142907:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.172687:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973723 is committed 00002000:00010000:3.0:1789760788.172823:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26238976 -> 26243072 00010000:00010000:3.0:1789760788.172839:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.172851:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.172860:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.172867:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.172872:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.172881:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.172904:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.172912:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20194 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.173173:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515821952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30367us (30783us total) trans 4294973723 rc 0/0 00000100:00100000:3.0:1789760788.173185:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6435 00000100:00100000:3.0:1789760788.173195:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766569621164 time 766608076067 state 0x2 ref 0 00000100:00100000:0.0:1789760788.181741:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.181823:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822080 00000100:00100000:1.0:1789760788.181833:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822080 00000100:00100000:1.0:1789760788.181890:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766569621164 time 766608076067 state 0x0 ref 1 00000100:00100000:1.0:1789760788.181904:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766608336483 time 766608076067 state 0x1 ref 1 00000100:00100000:1.0:1789760788.181933:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766608336483 time 766647221474 state 0x0 ref 1 00000100:00100000:1.0:1789760788.181949:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515822080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760788.182003:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26243072, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760788.182013:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.182027:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760788.182032:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.182038:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.182052:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.215625:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973724 is committed 00002000:00010000:2.0:1789760788.215784:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26243072 -> 26247168 00010000:00010000:2.0:1789760788.215800:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.215813:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.215822:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.215830:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760788.215835:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.215844:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.215864:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.215872:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d2019b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.216005:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515822080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34056us (34267us total) trans 4294973724 rc 0/0 00000100:00100000:2.0:1789760788.216015:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6436 00000100:00100000:2.0:1789760788.216024:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766608336483 time 766647221474 state 0x2 ref 0 00000100:00100000:0.0:1789760788.219646:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.219807:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822208 00000100:00100000:2.0:1789760788.219817:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822208 00000100:00100000:2.0:1789760788.219868:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766608336483 time 766647221474 state 0x0 ref 1 00000100:00100000:2.0:1789760788.219883:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766647481890 time 766647221474 state 0x1 ref 1 00000100:00100000:2.0:1789760788.219949:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766647481890 time 766685237244 state 0x0 ref 1 00000100:00100000:2.0:1789760788.219961:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515822208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760788.220007:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26247168, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:2.0:1789760788.220015:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.220031:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760788.220035:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.220040:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.220054:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.242254:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26247168 -> 26251264 00010000:00010000:3.0:1789760788.242273:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.242361:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.242371:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.242379:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.242384:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.242394:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.242413:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.242422:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.242513:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515822208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22549us (22881us total) trans 4294973725 rc 0/0 00000100:00100000:3.0:1789760788.242528:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6437 00000100:00100000:3.0:1789760788.242539:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766647481890 time 766685237244 state 0x2 ref 0 00000100:00100000:1.0:1789760788.245895:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:1.0:1789760788.245933:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973725 is committed 00000100:00100000:3.0:1789760788.246224:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822336 00000100:00100000:3.0:1789760788.246233:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822336 00000100:00100000:3.0:1789760788.246292:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766647481890 time 766685237244 state 0x0 ref 1 00000100:00100000:3.0:1789760788.246309:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766685497660 time 766685237244 state 0x1 ref 1 00000100:00100000:3.0:1789760788.246336:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766685497660 time 766711624872 state 0x0 ref 1 00000100:00100000:3.0:1789760788.246348:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515822336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760788.246391:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26251264, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:3.0:1789760788.246399:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.246415:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760788.246419:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.246425:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.246439:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.278425:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973726 is committed 00002000:00010000:0.0:1789760788.278518:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26251264 -> 26255360 00010000:00010000:0.0:1789760788.278534:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.278545:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.278565:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.278571:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760788.278576:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.278586:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.278604:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.278612:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d201a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.278738:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515822336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32360us (32822us total) trans 4294973726 rc 0/0 00000100:00100000:0.0:1789760788.278749:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6438 00000100:00100000:0.0:1789760788.278759:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766685497660 time 766711624872 state 0x2 ref 0 00000100:00100000:2.0:1789760788.282178:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760788.282269:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822464 00000100:00100000:0.0:1789760788.282338:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822464 00000100:00100000:0.0:1789760788.282395:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766685497660 time 766711624872 state 0x0 ref 1 00000100:00100000:0.0:1789760788.282409:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766711885288 time 766711624872 state 0x1 ref 1 00000100:00100000:0.0:1789760788.282472:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766711885288 time 766747760825 state 0x0 ref 1 00000100:00100000:0.0:1789760788.282485:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515822464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760788.282533:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26255360, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:0.0:1789760788.282542:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.282569:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760788.282574:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.282579:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.282593:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.312130:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26255360 -> 26259456 00010000:00010000:0.0:1789760788.312150:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.312164:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.312173:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.312181:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760788.312186:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.312195:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.312216:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.312225:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.312333:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515822464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29845us (30160us total) trans 4294973727 rc 0/0 00000100:00100000:0.0:1789760788.312344:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6439 00000100:00100000:0.0:1789760788.312354:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766711885288 time 766747760825 state 0x2 ref 0 00000001:00080000:1.0:1789760788.312486:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973727 is committed 00000100:00100000:2.0:1789760788.316357:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.316469:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822592 00000100:00100000:1.0:1789760788.316479:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822592 00000100:00100000:1.0:1789760788.316529:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766711885288 time 766747760825 state 0x0 ref 1 00000100:00100000:1.0:1789760788.316543:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766748021241 time 766747760825 state 0x1 ref 1 00000100:00100000:1.0:1789760788.316617:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766748021241 time 766781904581 state 0x0 ref 1 00000100:00100000:1.0:1789760788.316630:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515822592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760788.316690:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26259456, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760788.316700:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316744:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760788.316749:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.316756:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.316769:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.350765:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26259456 -> 26263552 00010000:00010000:1.0:1789760788.350787:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.350799:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.350808:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.350815:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.350819:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.350827:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.350848:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.350856:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d201b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.350941:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515822592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34309us (34591us total) trans 4294973728 rc 0/0 00000100:00100000:1.0:1789760788.350952:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6440 00000100:00100000:1.0:1789760788.350962:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766748021241 time 766781904581 state 0x2 ref 0 00000100:00100000:1.0:1789760788.354032:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760788.354115:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822720 00000100:00100000:3.0:1789760788.354124:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822720 00000100:00100000:3.0:1789760788.354176:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766748021241 time 766781904581 state 0x0 ref 1 00000100:00100000:3.0:1789760788.354192:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766782164997 time 766781904581 state 0x1 ref 1 00000100:00100000:3.0:1789760788.354224:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766782164997 time 766819512838 state 0x0 ref 1 00000100:00100000:3.0:1789760788.354237:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515822720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760788.354380:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26263552, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:3.0:1789760788.354390:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.354406:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760788.354410:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.354416:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.354430:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.355581:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973728 is committed 00000001:00080000:0.0:1789760788.383069:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973729 is committed 00002000:00010000:3.0:1789760788.383179:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26263552 -> 26267648 00010000:00010000:3.0:1789760788.383193:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.383206:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.383217:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.383225:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.383230:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.383239:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.383257:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.383265:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.383373:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515822720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29133us (29344us total) trans 4294973729 rc 0/0 00000100:00100000:3.0:1789760788.383856:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6441 00000100:00100000:3.0:1789760788.383867:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766782164997 time 766819512838 state 0x2 ref 0 00000100:00100000:3.0:1789760788.387808:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.388008:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822848 00000100:00100000:2.0:1789760788.388019:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822848 00000100:00100000:2.0:1789760788.388072:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766782164997 time 766819512838 state 0x0 ref 1 00000100:00100000:2.0:1789760788.388086:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766819773254 time 766819512838 state 0x1 ref 1 00000100:00100000:2.0:1789760788.388105:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766819773254 time 766853395174 state 0x0 ref 1 00000100:00100000:2.0:1789760788.388115:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515822848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760788.388161:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26267648, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:2.0:1789760788.388169:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.388184:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760788.388188:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.388195:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.388207:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.402121:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.402160:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.402185:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.402210:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823104 00000100:00100000:2.0:1789760788.402219:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823104 00000100:00100000:2.0:1789760788.402264:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 501 00000100:00100000:2.0:1789760788.402283:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515823104:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760788.402349:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2362:x1876699515823104:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 77us (237us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760788.402356:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 501 00000100:00100000:2.0:1789760788.402369:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823232 00000100:00100000:2.0:1789760788.402372:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823232 00000100:00100000:2.0:1789760788.402389:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 502 00000100:00100000:2.0:1789760788.402394:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515823232:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:2.0:1789760788.402413:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b109880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2362:x1876699515823232:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 19us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760788.402418:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 502 00000100:00080000:2.0:1789760788.402429:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760788.402494:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515822976 00000100:00100000:0.0:1789760788.402498:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515822976 00000100:00100000:0.0:1789760788.402525:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 410 00000100:00100000:0.0:1789760788.402530:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515822976:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:0.0:1789760788.402561:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2362:x1876699515822976:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 31us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760788.402567:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 410 00000001:00080000:0.0:1789760788.434503:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973730 is committed 00002000:00010000:3.0:1789760788.434551:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26267648 -> 26271744 00010000:00010000:3.0:1789760788.434567:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.434579:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.434594:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.434602:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.434607:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.434616:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.434644:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.434653:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d201c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.434785:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515822848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46665us (46980us total) trans 4294973730 rc 0/0 00000100:00100000:3.0:1789760788.434794:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6442 00000100:00100000:3.0:1789760788.434803:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766819773254 time 766853395174 state 0x2 ref 0 00000100:00100000:2.0:1789760788.438858:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760788.439538:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823360 00000100:00100000:3.0:1789760788.439549:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823360 00000100:00100000:3.0:1789760788.439606:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766819773254 time 766853395174 state 0x0 ref 1 00000100:00100000:3.0:1789760788.439621:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766853655590 time 766853395174 state 0x1 ref 1 00000100:00100000:3.0:1789760788.439661:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766853655590 time 766904950391 state 0x0 ref 1 00000100:00100000:3.0:1789760788.439674:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515823360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760788.439745:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26271744, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:3.0:1789760788.439754:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.439769:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760788.439774:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.439780:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.439795:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.482410:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973731 is committed 00002000:00010000:3.0:1789760788.482518:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26271744 -> 26275840 00010000:00010000:3.0:1789760788.482535:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482547:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482622:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482629:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.482643:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482652:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482671:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482678:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d201cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.482879:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515823360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43202us (44026us total) trans 4294973731 rc 0/0 00000100:00100000:3.0:1789760788.482891:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6443 00000100:00100000:3.0:1789760788.482901:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766853655590 time 766904950391 state 0x2 ref 0 00000100:00100000:1.0:1789760788.486092:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760788.486189:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823488 00000100:00100000:0.0:1789760788.486199:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823488 00000100:00100000:0.0:1789760788.486256:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766853655590 time 766904950391 state 0x0 ref 1 00000100:00100000:0.0:1789760788.486347:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766905210807 time 766904950391 state 0x1 ref 1 00000100:00100000:0.0:1789760788.486411:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766905210807 time 766951699524 state 0x0 ref 1 00000100:00100000:0.0:1789760788.486429:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515823488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760788.486487:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26275840, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:0.0:1789760788.486497:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.486513:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760788.486518:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.486524:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.486539:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.508907:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973732 is committed 00002000:00010000:1.0:1789760788.509001:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26275840 -> 26279936 00010000:00010000:1.0:1789760788.509017:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.509030:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.509039:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.509047:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.509052:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.509062:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.509081:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.509089:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d201d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.509178:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515823488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22747us (23092us total) trans 4294973732 rc 0/0 00000100:00100000:1.0:1789760788.509187:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6444 00000100:00100000:1.0:1789760788.509197:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766905210807 time 766951699524 state 0x2 ref 0 00000100:00100000:1.0:1789760788.513514:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760788.513650:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823616 00000100:00100000:0.0:1789760788.513661:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823616 00000100:00100000:0.0:1789760788.513742:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766905210807 time 766951699524 state 0x0 ref 1 00000100:00100000:0.0:1789760788.513757:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766951959940 time 766951699524 state 0x1 ref 1 00000100:00100000:0.0:1789760788.513782:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766951959940 time 766979072106 state 0x0 ref 1 00000100:00100000:0.0:1789760788.513794:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515823616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760788.513841:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26279936, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:0.0:1789760788.513850:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.513870:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760788.513875:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.513881:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.513895:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.539745:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973733 is committed 00002000:00010000:1.0:1789760788.539841:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26279936 -> 26284032 00010000:00010000:1.0:1789760788.539857:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.539870:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.539880:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.539895:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.539900:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.539910:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.539931:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.539939:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d201da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.540028:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515823616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26231us (26520us total) trans 4294973733 rc 0/0 00000100:00100000:1.0:1789760788.540038:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6445 00000100:00100000:1.0:1789760788.540049:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766951959940 time 766979072106 state 0x2 ref 0 00000100:00100000:3.0:1789760788.543077:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.543147:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823744 00000100:00100000:1.0:1789760788.543157:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823744 00000100:00100000:1.0:1789760788.543208:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766951959940 time 766979072106 state 0x0 ref 1 00000100:00100000:1.0:1789760788.543223:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766979332522 time 766979072106 state 0x1 ref 1 00000100:00100000:1.0:1789760788.543253:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766979332522 time 767008541981 state 0x0 ref 1 00000100:00100000:1.0:1789760788.543268:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515823744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760788.543384:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26284032, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760788.543393:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.543408:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.543412:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.543419:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.543433:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.574760:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973734 is committed 00002000:00010000:1.0:1789760788.574875:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26284032 -> 26288128 00010000:00010000:1.0:1789760788.574889:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.574902:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.574911:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.574918:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.574923:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.574933:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.574951:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.574958:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d201e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.575042:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515823744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31770us (31970us total) trans 4294973734 rc 0/0 00000100:00100000:1.0:1789760788.575052:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6446 00000100:00100000:1.0:1789760788.575061:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766979332522 time 767008541981 state 0x2 ref 0 00000100:00100000:3.0:1789760788.578510:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760788.578542:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515823872 00000100:00100000:3.0:1789760788.578629:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515823872 00000100:00100000:3.0:1789760788.578685:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 766979332522 time 767008541981 state 0x0 ref 1 00000100:00100000:3.0:1789760788.578701:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767008802397 time 767008541981 state 0x1 ref 1 00000100:00100000:3.0:1789760788.578828:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767008802397 time 767044116735 state 0x0 ref 1 00000100:00100000:3.0:1789760788.578840:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515823872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760788.578883:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26288128, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:3.0:1789760788.578892:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.578906:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.578911:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.578917:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.578931:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.601594:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26288128 -> 26292224 00010000:00010000:3.0:1789760788.601614:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601626:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601645:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601652:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.601657:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601666:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601686:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601693:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.601804:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515823872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22960us (23298us total) trans 4294973735 rc 0/0 00000100:00100000:3.0:1789760788.601815:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6447 00000100:00100000:3.0:1789760788.601825:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767008802397 time 767044116735 state 0x2 ref 0 00000001:00080000:2.0:1789760788.602700:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973735 is committed 00000100:00100000:3.0:1789760788.605035:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.605126:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824000 00000100:00100000:2.0:1789760788.605136:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824000 00000100:00100000:2.0:1789760788.605188:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767008802397 time 767044116735 state 0x0 ref 1 00000100:00100000:2.0:1789760788.605204:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767044377151 time 767044116735 state 0x1 ref 1 00000100:00100000:2.0:1789760788.605235:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767044377151 time 767070523827 state 0x0 ref 1 00000100:00100000:2.0:1789760788.605248:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760788.605308:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26292224, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:2.0:1789760788.605318:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.605333:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760788.605337:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.605343:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.605357:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.628563:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26292224 -> 26296320 00010000:00010000:3.0:1789760788.628583:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.628584:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973736 is committed 00010000:00010000:3.0:1789760788.628595:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.628604:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.628617:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.628622:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.628643:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.628663:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.628671:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d201ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.629221:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23967us (24187us total) trans 4294973736 rc 0/0 00000100:00100000:3.0:1789760788.629233:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6448 00000100:00100000:3.0:1789760788.629248:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767044377151 time 767070523827 state 0x2 ref 0 00000100:00100000:1.0:1789760788.638146:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760788.638247:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824128 00000100:00100000:3.0:1789760788.638257:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824128 00000100:00100000:3.0:1789760788.638311:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767044377151 time 767070523827 state 0x0 ref 1 00000100:00100000:3.0:1789760788.638327:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767070784243 time 767070523827 state 0x1 ref 1 00000100:00100000:3.0:1789760788.638569:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767070784243 time 767103856373 state 0x0 ref 1 00000100:00100000:3.0:1789760788.638587:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760788.638646:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26296320, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:3.0:1789760788.638655:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.638685:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.638690:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.638697:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.638746:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.683067:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973737 is committed 00002000:00010000:3.0:1789760788.683233:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26296320 -> 26300416 00010000:00010000:3.0:1789760788.683249:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.683261:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.683270:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.683276:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.683281:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.683290:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.683308:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.683315:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d201f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.683409:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44820us (45267us total) trans 4294973737 rc 0/0 00000100:00100000:3.0:1789760788.683418:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6449 00000100:00100000:3.0:1789760788.683427:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767070784243 time 767103856373 state 0x2 ref 0 00000100:00100000:1.0:1789760788.688257:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760788.688569:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824256 00000100:00100000:0.0:1789760788.688580:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824256 00000100:00100000:0.0:1789760788.688641:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767070784243 time 767103856373 state 0x0 ref 1 00000100:00100000:0.0:1789760788.688656:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767104116789 time 767103856373 state 0x1 ref 1 00000100:00100000:0.0:1789760788.688755:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767104116789 time 767154043262 state 0x0 ref 1 00000100:00100000:0.0:1789760788.688767:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760788.688813:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26300416, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:0.0:1789760788.688821:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.688837:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.688842:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.688848:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.688860:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.739426:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26300416 -> 26304512 00010000:00010000:1.0:1789760788.739447:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.739447:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973738 is committed 00010000:00010000:1.0:1789760788.739462:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.739473:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.739486:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760788.739491:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.739499:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.739518:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.739526:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d201fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.739795:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51024us (51540us total) trans 4294973738 rc 0/0 00000100:00100000:1.0:1789760788.739806:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6450 00000100:00100000:1.0:1789760788.739817:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767104116789 time 767154043262 state 0x2 ref 0 00000100:00100000:2.0:1789760788.748795:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.748832:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824384 00000100:00100000:1.0:1789760788.748843:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824384 00000100:00100000:1.0:1789760788.748901:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767104116789 time 767154043262 state 0x0 ref 1 00000100:00100000:1.0:1789760788.748917:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767154303678 time 767154043262 state 0x1 ref 1 00000100:00100000:1.0:1789760788.748949:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767154303678 time 767214237892 state 0x0 ref 1 00000100:00100000:1.0:1789760788.748964:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760788.749023:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26304512, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760788.749032:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.749049:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.749053:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.749069:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.749083:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.785905:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26304512 -> 26308608 00010000:00010000:3.0:1789760788.785925:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785938:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785948:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785961:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760788.785966:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785974:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785993:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.785999:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20204 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.786125:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515824384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37158us (37416us total) trans 4294973739 rc 0/0 00000100:00100000:3.0:1789760788.786135:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6451 00000100:00100000:3.0:1789760788.786145:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767154303678 time 767214237892 state 0x2 ref 0 00000001:00080000:3.0:1789760788.786182:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973739 is committed 00000100:00100000:2.0:1789760788.797189:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760788.797333:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824512 00000100:00100000:1.0:1789760788.797345:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824512 00000100:00100000:1.0:1789760788.797395:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767154303678 time 767214237892 state 0x0 ref 1 00000100:00100000:1.0:1789760788.797409:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767214498308 time 767214237892 state 0x1 ref 1 00000100:00100000:1.0:1789760788.797440:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767214498308 time 767262728762 state 0x0 ref 1 00000100:00100000:1.0:1789760788.797452:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760788.797501:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26308608, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:1.0:1789760788.797509:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.797530:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.797534:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.797540:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.797561:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.798079:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760788.850399:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973740 is committed 00002000:00010000:2.0:1789760788.850644:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26308608 -> 26312704 00010000:00010000:2.0:1789760788.850660:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.850681:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.850691:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.850704:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760788.850739:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.850749:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.850769:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.850776:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2020b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.850868:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53413us (53683us total) trans 4294973740 rc 0/0 00000100:00100000:2.0:1789760788.850880:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6452 00000100:00100000:2.0:1789760788.850890:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767214498308 time 767262728762 state 0x2 ref 0 00000100:00100000:1.0:1789760788.856128:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760788.856279:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824640 00000100:00100000:3.0:1789760788.856289:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824640 00000100:00100000:3.0:1789760788.856340:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767214498308 time 767262728762 state 0x0 ref 1 00000100:00100000:3.0:1789760788.856356:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767262989178 time 767262728762 state 0x1 ref 1 00000100:00100000:3.0:1789760788.856392:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767262989178 time 767321681161 state 0x0 ref 1 00000100:00100000:3.0:1789760788.856405:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515824640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760788.856454:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26312704, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd007 00010000:00010000:3.0:1789760788.856463:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.856479:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.856483:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.856489:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.856505:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.908197:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973741 is committed 00002000:00010000:2.0:1789760788.908306:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26312704 -> 26316800 00002000:00010000:2.0:1789760788.908312:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:2.0:1789760788.908314:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00010000:00010000:2.0:1789760788.908328:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.908341:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.908352:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.908360:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760788.908364:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.908373:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.908391:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.908400:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20212 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.908493:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515824640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52085us (52369us total) trans 4294973741 rc 0/0 00000100:00100000:2.0:1789760788.908503:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6453 00000100:00100000:2.0:1789760788.908512:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767262989178 time 767321681161 state 0x2 ref 0 00000100:00100000:3.0:1789760788.913586:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760788.913778:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824768 00000100:00100000:2.0:1789760788.913788:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824768 00000100:00100000:2.0:1789760788.913848:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767262989178 time 767321681161 state 0x0 ref 1 00000100:00100000:2.0:1789760788.913862:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767321941577 time 767321681161 state 0x1 ref 1 00000100:00100000:2.0:1789760788.913889:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767321941577 time 767379178580 state 0x0 ref 1 00000100:00100000:2.0:1789760788.913901:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515824768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760788.913947:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26316800, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:2.0:1789760788.913955:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.913971:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760788.913975:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.913982:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.913996:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.998358:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973742 is committed 00002000:00010000:2.0:1789760788.998976:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26316800 -> 26320896 00010000:00010000:2.0:1789760788.998992:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.999004:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.999013:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.999021:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760788.999026:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.999035:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.999055:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.999063:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20219 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.999402:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515824768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 85499us (85821us total) trans 4294973742 rc 0/0 00000100:00100000:2.0:1789760788.999413:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6454 00000100:00100000:2.0:1789760788.999423:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767321941577 time 767379178580 state 0x2 ref 0 00000100:00100000:3.0:1789760789.012160:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760789.012407:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515824896 00000100:00100000:2.0:1789760789.012417:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515824896 00000100:00100000:2.0:1789760789.012469:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767321941577 time 767379178580 state 0x0 ref 1 00000100:00100000:2.0:1789760789.012485:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767379438996 time 767379178580 state 0x1 ref 1 00000100:00100000:2.0:1789760789.012514:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767379438996 time 767477803364 state 0x0 ref 1 00000100:00100000:2.0:1789760789.012527:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760789.012578:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26320896, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:2.0:1789760789.012588:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.012603:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760789.012607:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.012613:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.012642:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.066694:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973743 is committed 00002000:00010000:2.0:1789760789.068114:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26320896 -> 26324992 00010000:00010000:2.0:1789760789.068133:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.068147:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.068156:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.068162:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760789.068167:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.068176:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.068201:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.068210:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d20220 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.068803:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515824896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56273us (56657us total) trans 4294973743 rc 0/0 00000100:00100000:2.0:1789760789.068815:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6455 00000100:00100000:2.0:1789760789.068826:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767379438996 time 767477803364 state 0x2 ref 0 00000100:00100000:2.0:1789760789.079182:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.079311:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825024 00000100:00100000:1.0:1789760789.079322:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825024 00000100:00100000:1.0:1789760789.079377:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767379438996 time 767477803364 state 0x0 ref 1 00000100:00100000:1.0:1789760789.079392:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767478063780 time 767477803364 state 0x1 ref 1 00000100:00100000:1.0:1789760789.079417:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767478063780 time 767544705849 state 0x0 ref 1 00000100:00100000:1.0:1789760789.079429:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515825024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.079475:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26324992, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.079483:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.079499:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760789.079503:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.079510:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.079524:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760789.155009:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26324992 -> 26329088 00010000:00010000:2.0:1789760789.155030:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.155044:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.155053:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.155061:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760789.155066:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.155080:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.155101:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.155108:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20227 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.155205:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515825024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 75774us (76028us total) trans 4294973744 rc 0/0 00000100:00100000:2.0:1789760789.155216:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6456 00000100:00100000:2.0:1789760789.155225:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767478063780 time 767544705849 state 0x2 ref 0 00000001:00080000:3.0:1789760789.155379:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973744 is committed 00000100:00100000:3.0:1789760789.159952:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.160162:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825152 00000100:00100000:1.0:1789760789.160174:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825152 00000100:00100000:1.0:1789760789.160239:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767478063780 time 767544705849 state 0x0 ref 1 00000100:00100000:1.0:1789760789.160255:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767544966265 time 767544705849 state 0x1 ref 1 00000100:00100000:1.0:1789760789.160284:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767544966265 time 767625573251 state 0x0 ref 1 00000100:00100000:1.0:1789760789.160297:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515825152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.160347:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26329088, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.160356:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.160372:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760789.160376:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.160383:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.160396:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.193210:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26329088 -> 26333184 00010000:00010000:0.0:1789760789.193240:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.193252:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.193261:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.193268:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760789.193272:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.193280:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.193300:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.193309:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d2022e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.193402:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515825152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33102us (33456us total) trans 4294973745 rc 0/0 00000100:00100000:0.0:1789760789.193412:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6457 00000100:00100000:0.0:1789760789.193421:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767544966265 time 767625573251 state 0x2 ref 0 00000001:00080000:2.0:1789760789.195108:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973745 is committed 00000100:00100000:0.0:1789760789.199018:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.199108:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825280 00000100:00100000:1.0:1789760789.199117:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825280 00000100:00100000:1.0:1789760789.199168:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767544966265 time 767625573251 state 0x0 ref 1 00000100:00100000:1.0:1789760789.199183:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767625833667 time 767625573251 state 0x1 ref 1 00000100:00100000:1.0:1789760789.199216:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767625833667 time 767664505524 state 0x0 ref 1 00000100:00100000:1.0:1789760789.199234:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515825280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.199281:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26333184, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.199291:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.199307:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760789.199312:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.199318:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.199333:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.311025:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973746 is committed 00002000:00010000:3.0:1789760789.311278:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26333184 -> 26337280 00010000:00010000:3.0:1789760789.311294:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.311308:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.311318:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.311325:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.311329:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.311338:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.311357:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.311364:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d20235 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.311452:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515825280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 112221us (112440us total) trans 4294973746 rc 0/0 00000100:00100000:3.0:1789760789.311462:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6458 00000100:00100000:3.0:1789760789.311471:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767625833667 time 767664505524 state 0x2 ref 0 00000100:00100000:3.0:1789760789.314249:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760789.314450:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825408 00000100:00100000:2.0:1789760789.314459:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825408 00000100:00100000:2.0:1789760789.314513:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767625833667 time 767664505524 state 0x0 ref 1 00000100:00100000:2.0:1789760789.314529:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767664765940 time 767664505524 state 0x1 ref 1 00000100:00100000:2.0:1789760789.314569:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767664765940 time 767779858147 state 0x0 ref 1 00000100:00100000:2.0:1789760789.314581:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515825408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760789.314634:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26337280, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:2.0:1789760789.314643:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.314659:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760789.314663:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.314668:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.314683:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.361756:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26337280 -> 26341376 00010000:00010000:3.0:1789760789.361776:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.361795:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.361805:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.361813:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00000001:00080000:2.0:1789760789.361816:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973747 is committed 00010000:00010000:3.0:1789760789.361818:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.361829:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.361849:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.361857:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d2023c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.361944:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515825408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47360us (47700us total) trans 4294973747 rc 0/0 00000100:00100000:3.0:1789760789.361954:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6459 00000100:00100000:3.0:1789760789.361964:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767664765940 time 767779858147 state 0x2 ref 0 00000100:00100000:2.0:1789760789.364815:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760789.364927:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825536 00000100:00100000:0.0:1789760789.364937:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825536 00000100:00100000:0.0:1789760789.364985:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767664765940 time 767779858147 state 0x0 ref 1 00000100:00100000:0.0:1789760789.364998:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767780118563 time 767779858147 state 0x1 ref 1 00000100:00100000:0.0:1789760789.365027:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767780118563 time 767830316276 state 0x0 ref 1 00000100:00100000:0.0:1789760789.365038:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515825536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760789.365081:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26341376, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:0.0:1789760789.365089:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.365104:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760789.365108:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.365114:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.365128:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.423976:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973748 is committed 00002000:00010000:1.0:1789760789.424383:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26341376 -> 26345472 00010000:00010000:1.0:1789760789.424400:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.424412:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.424426:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.424433:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760789.424438:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.424447:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.424467:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.424476:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20243 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.424660:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515825536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59618us (59852us total) trans 4294973748 rc 0/0 00000100:00100000:1.0:1789760789.424671:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6460 00000100:00100000:1.0:1789760789.424680:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767780118563 time 767830316276 state 0x2 ref 0 00000100:00100000:0.0:1789760789.428413:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760789.428549:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825664 00000100:00100000:2.0:1789760789.428559:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825664 00000100:00100000:2.0:1789760789.428612:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767780118563 time 767830316276 state 0x0 ref 1 00000100:00100000:2.0:1789760789.428642:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767830576692 time 767830316276 state 0x1 ref 1 00000100:00100000:2.0:1789760789.428742:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767830576692 time 767894029987 state 0x0 ref 1 00000100:00100000:2.0:1789760789.428756:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515825664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760789.428805:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26345472, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:2.0:1789760789.428814:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.428835:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760789.428839:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.428845:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.428858:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.474921:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973749 is committed 00002000:00010000:2.0:1789760789.476513:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26345472 -> 26349568 00010000:00010000:2.0:1789760789.476532:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.476545:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.476552:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.476559:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760789.476563:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.476572:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.476590:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.476598:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2024a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.476703:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515825664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47944us (48295us total) trans 4294973749 rc 0/0 00000100:00100000:2.0:1789760789.476763:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6461 00000100:00100000:2.0:1789760789.476773:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767830576692 time 767894029987 state 0x2 ref 0 00000100:00100000:3.0:1789760789.483679:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.484269:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825792 00000100:00100000:1.0:1789760789.484280:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825792 00000100:00100000:1.0:1789760789.484333:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767830576692 time 767894029987 state 0x0 ref 1 00000100:00100000:1.0:1789760789.484363:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767894290403 time 767894029987 state 0x1 ref 1 00000100:00100000:1.0:1789760789.484395:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767894290403 time 767949683890 state 0x0 ref 1 00000100:00100000:1.0:1789760789.484408:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515825792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.484456:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26349568, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.484466:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.484482:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760789.484486:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.484493:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.484507:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.531162:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973750 is committed 00002000:00010000:3.0:1789760789.531338:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26349568 -> 26353664 00010000:00010000:3.0:1789760789.531353:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.531366:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.531375:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.531382:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.531386:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.531394:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.531414:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.531421:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20251 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.531517:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515825792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47107us (47842us total) trans 4294973750 rc 0/0 00000100:00100000:3.0:1789760789.531531:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6462 00000100:00100000:3.0:1789760789.531541:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767894290403 time 767949683890 state 0x2 ref 0 00000100:00100000:3.0:1789760789.535897:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760789.536098:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515825920 00000100:00100000:0.0:1789760789.536110:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515825920 00000100:00100000:0.0:1789760789.536159:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767894290403 time 767949683890 state 0x0 ref 1 00000100:00100000:0.0:1789760789.536173:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767949944306 time 767949683890 state 0x1 ref 1 00000100:00100000:0.0:1789760789.536205:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767949944306 time 768001494329 state 0x0 ref 1 00000100:00100000:0.0:1789760789.536216:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515825920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760789.536272:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26353664, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:0.0:1789760789.536289:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.536304:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.536309:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.536314:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.536330:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.582140:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26353664 -> 26357760 00010000:00010000:0.0:1789760789.582160:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.582178:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.582187:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.582194:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bda00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760789.582199:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.582208:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.582227:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.582236:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b07b88c2/0x69dc3c3d61d20258 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.582670:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973751 is committed 00000100:00100000:0.0:1789760789.583054:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515825920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46832us (47160us total) trans 4294973751 rc 0/0 00000100:00100000:0.0:1789760789.583065:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6463 00000100:00100000:0.0:1789760789.583075:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767949944306 time 768001494329 state 0x2 ref 0 00000100:00100000:1.0:1789760789.586421:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760789.586704:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826048 00000100:00100000:3.0:1789760789.586747:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826048 00000100:00100000:3.0:1789760789.586799:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 767949944306 time 768001494329 state 0x0 ref 1 00000100:00100000:3.0:1789760789.586812:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768001754745 time 768001494329 state 0x1 ref 1 00000100:00100000:3.0:1789760789.586843:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768001754745 time 768052132038 state 0x0 ref 1 00000100:00100000:3.0:1789760789.586855:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515826048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760789.586901:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26357760, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:3.0:1789760789.586910:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.586925:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.586928:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.586935:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.586948:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.616178:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26357760 -> 26361856 00010000:00010000:3.0:1789760789.616198:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.616213:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.616222:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.616230:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.616234:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.616244:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.616263:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.616500:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2025f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.616605:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515826048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29747us (30188us total) trans 4294973752 rc 0/0 00000100:00100000:3.0:1789760789.616615:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6464 00000100:00100000:3.0:1789760789.616632:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768001754745 time 768052132038 state 0x2 ref 0 00000001:00080000:3.0:1789760789.616675:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973752 is committed 00000100:00100000:3.0:1789760789.620026:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.620127:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826176 00000100:00100000:1.0:1789760789.620137:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826176 00000100:00100000:1.0:1789760789.620193:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768001754745 time 768052132038 state 0x0 ref 1 00000100:00100000:1.0:1789760789.620209:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768052392454 time 768052132038 state 0x1 ref 1 00000100:00100000:1.0:1789760789.620310:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768052392454 time 768085598044 state 0x0 ref 1 00000100:00100000:1.0:1789760789.620325:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515826176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.620369:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26361856, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.620377:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.620393:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.620397:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.620403:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.620416:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.643519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973753 is committed 00002000:00010000:3.0:1789760789.643756:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26361856 -> 26365952 00010000:00010000:3.0:1789760789.643774:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.643786:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.643796:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.643804:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.643808:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.643818:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.643838:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.643847:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20266 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.643946:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515826176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 23616us (23923us total) trans 4294973753 rc 0/0 00000100:00100000:3.0:1789760789.643957:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6465 00000100:00100000:3.0:1789760789.643968:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768052392454 time 768085598044 state 0x2 ref 0 00000100:00100000:3.0:1789760789.646903:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.646980:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826304 00000100:00100000:1.0:1789760789.646992:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826304 00000100:00100000:1.0:1789760789.647045:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768052392454 time 768085598044 state 0x0 ref 1 00000100:00100000:1.0:1789760789.647060:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768085858460 time 768085598044 state 0x1 ref 1 00000100:00100000:1.0:1789760789.647079:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768085858460 time 768112369236 state 0x0 ref 1 00000100:00100000:1.0:1789760789.647092:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515826304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.647139:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26365952, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.647148:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647170:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.647175:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.647181:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647195:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.681115:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973754 is committed 00002000:00010000:3.0:1789760789.681199:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26365952 -> 26370048 00010000:00010000:3.0:1789760789.681217:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.681230:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.681240:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.681248:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.681253:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.681261:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.681361:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.681369:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2026d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.681506:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515826304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34411us (34608us total) trans 4294973754 rc 0/0 00000100:00100000:3.0:1789760789.681516:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6466 00000100:00100000:3.0:1789760789.681525:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768085858460 time 768112369236 state 0x2 ref 0 00000100:00100000:1.0:1789760789.685699:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760789.685967:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826432 00000100:00100000:2.0:1789760789.685977:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826432 00000100:00100000:2.0:1789760789.686030:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768085858460 time 768112369236 state 0x0 ref 1 00000100:00100000:2.0:1789760789.686048:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768112629652 time 768112369236 state 0x1 ref 1 00000100:00100000:2.0:1789760789.686081:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768112629652 time 768151369912 state 0x0 ref 1 00000100:00100000:2.0:1789760789.686093:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515826432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760789.686143:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26370048, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:2.0:1789760789.686153:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.686169:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760789.686175:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.686181:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.686196:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.716104:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973755 is committed 00002000:00010000:3.0:1789760789.716209:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26370048 -> 26374144 00010000:00010000:3.0:1789760789.716224:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.716237:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.716248:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.716256:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.716261:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.716282:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.716301:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.716314:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d20274 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.716408:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515826432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30311us (30713us total) trans 4294973755 rc 0/0 00000100:00100000:3.0:1789760789.716419:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6467 00000100:00100000:3.0:1789760789.716428:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768112629652 time 768151369912 state 0x2 ref 0 00000100:00100000:2.0:1789760789.719643:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760789.719750:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826560 00000100:00100000:3.0:1789760789.719759:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826560 00000100:00100000:3.0:1789760789.719810:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768112629652 time 768151369912 state 0x0 ref 1 00000100:00100000:3.0:1789760789.719824:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768151630328 time 768151369912 state 0x1 ref 1 00000100:00100000:3.0:1789760789.719850:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768151630328 time 768185140414 state 0x0 ref 1 00000100:00100000:3.0:1789760789.719861:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515826560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760789.719907:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26374144, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:3.0:1789760789.719917:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.719932:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.719937:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.719943:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.719963:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.751475:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973756 is committed 00002000:00010000:3.0:1789760789.751597:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26374144 -> 26378240 00010000:00010000:3.0:1789760789.751613:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.751637:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.751646:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.751654:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.751658:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.751667:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.751686:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.751693:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2027b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.751809:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515826560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31944us (32181us total) trans 4294973756 rc 0/0 00000100:00100000:3.0:1789760789.751820:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6468 00000100:00100000:3.0:1789760789.751830:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768151630328 time 768185140414 state 0x2 ref 0 00000100:00100000:3.0:1789760789.759171:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.760070:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826688 00000100:00100000:1.0:1789760789.760084:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826688 00000100:00100000:1.0:1789760789.760137:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768151630328 time 768185140414 state 0x0 ref 1 00000100:00100000:1.0:1789760789.760152:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768185400830 time 768185140414 state 0x1 ref 1 00000100:00100000:1.0:1789760789.760181:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768185400830 time 768225470681 state 0x0 ref 1 00000100:00100000:1.0:1789760789.760191:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515826688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.760235:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26378240, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:1.0:1789760789.760243:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.760257:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.760261:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.760277:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.760291:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.788946:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973757 is committed 00002000:00010000:0.0:1789760789.789045:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26378240 -> 26382336 00010000:00010000:0.0:1789760789.789061:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.789075:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.789084:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.789091:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760789.789097:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.789106:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.789125:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.789133:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20282 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.789225:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515826688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29030us (30058us total) trans 4294973757 rc 0/0 00000100:00100000:0.0:1789760789.789240:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6469 00000100:00100000:0.0:1789760789.789249:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768185400830 time 768225470681 state 0x2 ref 0 00000100:00100000:0.0:1789760789.793100:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760789.793188:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826816 00000100:00100000:3.0:1789760789.793199:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826816 00000100:00100000:3.0:1789760789.793250:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768185400830 time 768225470681 state 0x0 ref 1 00000100:00100000:3.0:1789760789.793333:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768225731097 time 768225470681 state 0x1 ref 1 00000100:00100000:3.0:1789760789.793401:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768225731097 time 768258689074 state 0x0 ref 1 00000100:00100000:3.0:1789760789.793413:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515826816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760789.793459:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26382336, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:3.0:1789760789.793467:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.793482:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.793486:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.793493:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.793506:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.822219:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760789.842244:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973758 is committed 00002000:00010000:0.0:1789760789.842395:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26382336 -> 26386432 00010000:00010000:0.0:1789760789.842409:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842421:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842430:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842438:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760789.842442:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842451:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842469:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842476:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20289 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.842687:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515826816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49270us (49592us total) trans 4294973758 rc 0/0 00000100:00100000:0.0:1789760789.842698:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6470 00000100:00100000:0.0:1789760789.842748:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768225731097 time 768258689074 state 0x2 ref 0 00000100:00100000:2.0:1789760789.845918:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760789.845993:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515826944 00000100:00100000:0.0:1789760789.846002:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515826944 00000100:00100000:0.0:1789760789.846050:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768225731097 time 768258689074 state 0x0 ref 1 00000100:00100000:0.0:1789760789.846065:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768258949490 time 768258689074 state 0x1 ref 1 00000100:00100000:0.0:1789760789.846091:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768258949490 time 768311380994 state 0x0 ref 1 00000100:00100000:0.0:1789760789.846102:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515826944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760789.846145:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26386432, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd007 00010000:00010000:0.0:1789760789.846154:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.846169:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.846174:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.846179:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.846193:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.876450:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973759 is committed 00002000:00010000:0.0:1789760789.876583:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26386432 -> 26390528 00002000:00010000:0.0:1789760789.876588:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:0.0:1789760789.876590:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00010000:00010000:0.0:1789760789.876603:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.876614:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.876633:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.876641:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760789.876645:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.876653:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.876671:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.876679:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20290 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.876803:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515826944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30696us (30886us total) trans 4294973759 rc 0/0 00000100:00100000:0.0:1789760789.876814:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6471 00000100:00100000:0.0:1789760789.876823:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768258949490 time 768311380994 state 0x2 ref 0 00000100:00100000:3.0:1789760789.880571:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760789.881149:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827072 00000100:00100000:0.0:1789760789.881159:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827072 00000100:00100000:0.0:1789760789.881212:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768258949490 time 768311380994 state 0x0 ref 1 00000100:00100000:0.0:1789760789.881227:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768311641410 time 768311380994 state 0x1 ref 1 00000100:00100000:0.0:1789760789.881349:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768311641410 time 768346636685 state 0x0 ref 1 00000100:00100000:0.0:1789760789.881362:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515827072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760789.881408:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26390528, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760789.881417:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.881433:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.881437:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.881443:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.881455:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.909007:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26390528 -> 26394624 00010000:00010000:3.0:1789760789.909030:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.909045:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.909055:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.909061:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.909066:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.909074:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.909093:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.909101:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20297 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.909201:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515827072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27836us (28634us total) trans 4294973760 rc 0/0 00000100:00100000:3.0:1789760789.909211:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6472 00000100:00100000:3.0:1789760789.909221:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768311641410 time 768346636685 state 0x2 ref 0 00000001:00080000:2.0:1789760789.910065:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973760 is committed 00000100:00100000:0.0:1789760789.913025:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760789.913118:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827200 00000100:00100000:2.0:1789760789.913128:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827200 00000100:00100000:2.0:1789760789.913182:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768311641410 time 768346636685 state 0x0 ref 1 00000100:00100000:2.0:1789760789.913198:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768346897101 time 768346636685 state 0x1 ref 1 00000100:00100000:2.0:1789760789.913229:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768346897101 time 768378518042 state 0x0 ref 1 00000100:00100000:2.0:1789760789.913241:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515827200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760789.913301:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26394624, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760789.913311:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.913327:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760789.913332:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.913338:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.913351:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.937417:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973761 is committed 00002000:00010000:2.0:1789760789.937609:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26394624 -> 26398720 00010000:00010000:2.0:1789760789.937635:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.937649:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.937659:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.937666:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760789.937671:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.937680:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.937701:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.937743:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d2029e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.937832:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515827200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24587us (24811us total) trans 4294973761 rc 0/0 00000100:00100000:2.0:1789760789.937845:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6473 00000100:00100000:2.0:1789760789.937857:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768346897101 time 768378518042 state 0x2 ref 0 00000100:00100000:0.0:1789760789.941885:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760789.941961:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827328 00000100:00100000:3.0:1789760789.941972:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827328 00000100:00100000:3.0:1789760789.942026:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768346897101 time 768378518042 state 0x0 ref 1 00000100:00100000:3.0:1789760789.942040:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768378778458 time 768378518042 state 0x1 ref 1 00000100:00100000:3.0:1789760789.942071:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768378778458 time 768407360589 state 0x0 ref 1 00000100:00100000:3.0:1789760789.942083:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515827328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760789.942127:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26398720, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:3.0:1789760789.942136:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.942152:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.942156:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.942163:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.942177:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.971871:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26398720 -> 26402816 00010000:00010000:3.0:1789760789.971893:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.971908:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.971919:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.971927:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760789.971933:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.971943:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.971968:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.971975:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d202a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.972069:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515827328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29982us (30190us total) trans 4294973762 rc 0/0 00000100:00100000:3.0:1789760789.972085:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6474 00000100:00100000:3.0:1789760789.972095:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768378778458 time 768407360589 state 0x2 ref 0 00000001:00080000:3.0:1789760789.973188:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973762 is committed 00000100:00100000:0.0:1789760789.975235:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760789.975370:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827456 00000100:00100000:1.0:1789760789.975380:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827456 00000100:00100000:1.0:1789760789.975433:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768378778458 time 768407360589 state 0x0 ref 1 00000100:00100000:1.0:1789760789.975449:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768407621005 time 768407360589 state 0x1 ref 1 00000100:00100000:1.0:1789760789.975478:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768407621005 time 768440768383 state 0x0 ref 1 00000100:00100000:1.0:1789760789.975490:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515827456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760789.975606:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26402816, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:1.0:1789760789.975618:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.975643:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.975647:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.975654:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.975669:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.006837:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973763 is committed 00002000:00010000:0.0:1789760790.006937:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26402816 -> 26406912 00010000:00010000:0.0:1789760790.006955:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.006967:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.006976:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.006983:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760790.006988:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.006996:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.007015:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.007022:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d202ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.007119:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515827456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31626us (31889us total) trans 4294973763 rc 0/0 00000100:00100000:0.0:1789760790.007129:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6475 00000100:00100000:0.0:1789760790.007138:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768407621005 time 768440768383 state 0x2 ref 0 00000100:00100000:3.0:1789760790.011747:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.012001:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827584 00000100:00100000:0.0:1789760790.012012:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827584 00000100:00100000:0.0:1789760790.012075:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768407621005 time 768440768383 state 0x0 ref 1 00000100:00100000:0.0:1789760790.012090:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768441028799 time 768440768383 state 0x1 ref 1 00000100:00100000:0.0:1789760790.012125:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768441028799 time 768477413718 state 0x0 ref 1 00000100:00100000:0.0:1789760790.012137:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515827584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.012187:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26406912, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760790.012195:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.012211:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760790.012215:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.012222:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.012236:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760790.012805:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760790.016085:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544421888:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760790.016170:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760790.016258:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544421888 00000100:00100000:2.0:1789760790.016266:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544421888 00000100:00100000:2.0:1789760790.016323:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1789760790.016331:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544421888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760790.016351:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3831496704 avail 3815768064 00000100:00100000:2.0:1789760790.016394:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544421888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760790.016401:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00080000:2.0:1789760790.016425:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760790.016461:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544421888:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760790.016886:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804c4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544422016:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760790.016915:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760790.016997:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422016 00000100:00100000:1.0:1789760790.017002:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422016 00000100:00100000:1.0:1789760790.017046:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789760790.017053:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544422016:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760790.017079:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760790.017113:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544422016:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760790.017120:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00080000:2.0:1789760790.017170:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760790.017192:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804c4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544422016:0@lo:13:osp-pre-1-0.0 00002000:00010000:0.0:1789760790.042972:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26406912 -> 26411008 00010000:00010000:0.0:1789760790.042992:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.043006:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.043015:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.043023:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760790.043028:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.043037:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.043057:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.043064:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d202b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.043159:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515827584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31019us (31453us total) trans 4294973764 rc 0/0 00000100:00100000:0.0:1789760790.043173:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6476 00000100:00100000:0.0:1789760790.043182:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768441028799 time 768477413718 state 0x2 ref 0 00000001:00080000:1.0:1789760790.043263:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973764 is committed 00000100:00100000:1.0:1789760790.047136:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.047217:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827712 00000100:00100000:0.0:1789760790.047227:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827712 00000100:00100000:0.0:1789760790.047292:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768441028799 time 768477413718 state 0x0 ref 1 00000100:00100000:0.0:1789760790.047307:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768477674134 time 768477413718 state 0x1 ref 1 00000100:00100000:0.0:1789760790.047337:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768477674134 time 768512626550 state 0x0 ref 1 00000100:00100000:0.0:1789760790.047350:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515827712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.047396:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26411008, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760790.047405:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.047422:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760790.047426:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.047433:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.047446:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.069233:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973765 is committed 00002000:00010000:0.0:1789760790.069422:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26411008 -> 26415104 00010000:00010000:0.0:1789760790.069437:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.069451:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.069461:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.069468:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760790.069472:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.069481:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.069499:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.069507:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d202ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.069600:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515827712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 22247us (22468us total) trans 4294973765 rc 0/0 00000100:00100000:0.0:1789760790.069610:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6477 00000100:00100000:0.0:1789760790.069629:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768477674134 time 768512626550 state 0x2 ref 0 00000100:00100000:2.0:1789760790.073467:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.073610:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827840 00000100:00100000:3.0:1789760790.073629:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827840 00000100:00100000:3.0:1789760790.073679:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768477674134 time 768512626550 state 0x0 ref 1 00000100:00100000:3.0:1789760790.073695:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768512886966 time 768512626550 state 0x1 ref 1 00000100:00100000:2.0:1789760790.073922:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768512886966 time 768539208869 state 0x0 ref 1 00000100:00100000:2.0:1789760790.073937:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515827840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760790.073987:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26415104, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760790.073996:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.074011:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760790.074016:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.074022:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.074036:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.119031:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973766 is committed 00002000:00010000:1.0:1789760790.119193:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26415104 -> 26419200 00010000:00010000:1.0:1789760790.119208:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.119222:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.119231:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.119238:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.119243:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.119253:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.119337:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.119346:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.119426:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515827840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45489us (45964us total) trans 4294973766 rc 0/0 00000100:00100000:1.0:1789760790.119436:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6478 00000100:00100000:1.0:1789760790.119445:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768512886966 time 768539208869 state 0x2 ref 0 00000100:00100000:0.0:1789760790.122430:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.122754:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515827968 00000100:00100000:3.0:1789760790.122764:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515827968 00000100:00100000:3.0:1789760790.122816:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768512886966 time 768539208869 state 0x0 ref 1 00000100:00100000:3.0:1789760790.122832:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768539469285 time 768539208869 state 0x1 ref 1 00000100:00100000:3.0:1789760790.122851:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768539469285 time 768588141482 state 0x0 ref 1 00000100:00100000:3.0:1789760790.122863:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515827968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760790.122911:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26419200, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:3.0:1789760790.122920:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.122941:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760790.122946:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.122952:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.122965:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760790.154763:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26419200 -> 26423296 00010000:00010000:3.0:1789760790.154783:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.154796:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.154806:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.154813:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760790.154818:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.154828:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.154844:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.154852:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.154937:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515827968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32071us (32512us total) trans 4294973767 rc 0/0 00000100:00100000:3.0:1789760790.154946:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6479 00000100:00100000:3.0:1789760790.154955:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768539469285 time 768588141482 state 0x2 ref 0 00000100:00100000:1.0:1789760790.158252:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000001:00080000:0.0:1789760790.158410:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973767 is committed 00000100:00100000:3.0:1789760790.158497:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828096 00000100:00100000:3.0:1789760790.158508:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828096 00000100:00100000:2.0:1789760790.158682:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768539469285 time 768588141482 state 0x0 ref 1 00000100:00100000:2.0:1789760790.158697:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768588401898 time 768588141482 state 0x1 ref 1 00000100:00100000:2.0:1789760790.158778:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768588401898 time 768624066509 state 0x0 ref 1 00000100:00100000:2.0:1789760790.158791:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760790.158839:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26423296, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760790.158849:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.158865:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760790.158869:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.158876:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.158889:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.184031:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26423296 -> 26427392 00010000:00010000:2.0:1789760790.184053:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.184067:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.184077:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.184085:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.184090:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.184100:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.184120:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.184128:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d202cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.184219:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515828096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25423us (25968us total) trans 4294973768 rc 0/0 00000100:00100000:2.0:1789760790.184229:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6480 00000100:00100000:2.0:1789760790.184239:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768588401898 time 768624066509 state 0x2 ref 0 00000001:00080000:0.0:1789760790.184541:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973768 is committed 00000100:00100000:1.0:1789760790.187744:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.187853:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828224 00000100:00100000:0.0:1789760790.187864:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828224 00000100:00100000:0.0:1789760790.187915:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768588401898 time 768624066509 state 0x0 ref 1 00000100:00100000:0.0:1789760790.187930:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768624326925 time 768624066509 state 0x1 ref 1 00000100:00100000:0.0:1789760790.187958:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768624326925 time 768653247416 state 0x0 ref 1 00000100:00100000:0.0:1789760790.187970:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.188017:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26427392, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760790.188026:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.188043:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760790.188047:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.188053:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.188066:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.223996:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973769 is committed 00002000:00010000:1.0:1789760790.224136:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26427392 -> 26431488 00010000:00010000:1.0:1789760790.224152:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.224166:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.224176:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.224183:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.224189:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.224200:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.224219:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.224227:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d202d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.224326:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36353us (36588us total) trans 4294973769 rc 0/0 00000100:00100000:1.0:1789760790.224336:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6481 00000100:00100000:1.0:1789760790.224344:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768624326925 time 768653247416 state 0x2 ref 0 00000100:00100000:3.0:1789760790.229330:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760790.229397:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828352 00000100:00100000:1.0:1789760790.229406:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828352 00000100:00100000:1.0:1789760790.229458:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768624326925 time 768653247416 state 0x0 ref 1 00000100:00100000:1.0:1789760790.229474:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768653507832 time 768653247416 state 0x1 ref 1 00000100:00100000:1.0:1789760790.229517:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768653507832 time 768694806259 state 0x0 ref 1 00000100:00100000:1.0:1789760790.229529:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760790.229585:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26431488, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:1.0:1789760790.229594:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.229610:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760790.229614:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.229632:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.229646:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.278185:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973770 is committed 00002000:00010000:1.0:1789760790.278363:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26431488 -> 26435584 00010000:00010000:1.0:1789760790.278380:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.278394:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.278404:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.278411:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.278416:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.278427:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.278447:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.278456:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d202dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.278691:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49159us (49366us total) trans 4294973770 rc 0/0 00000100:00100000:1.0:1789760790.278701:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6482 00000100:00100000:1.0:1789760790.279645:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768653507832 time 768694806259 state 0x2 ref 0 00000100:00100000:0.0:1789760790.282829:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760790.283030:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828480 00000100:00100000:2.0:1789760790.283040:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828480 00000100:00100000:2.0:1789760790.283090:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768653507832 time 768694806259 state 0x0 ref 1 00000100:00100000:2.0:1789760790.283104:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768695066675 time 768694806259 state 0x1 ref 1 00000100:00100000:2.0:1789760790.283168:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768695066675 time 768748457365 state 0x0 ref 1 00000100:00100000:2.0:1789760790.283180:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515828480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760790.283225:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26435584, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760790.283234:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.283250:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760790.283253:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.283270:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.283284:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.314884:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973771 is committed 00002000:00010000:2.0:1789760790.315158:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26435584 -> 26439680 00010000:00010000:2.0:1789760790.315174:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.315188:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.315198:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.315206:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.315211:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.315221:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.315241:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.315249:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.315425:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515828480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32239us (32599us total) trans 4294973771 rc 0/0 00000100:00100000:2.0:1789760790.315439:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6483 00000100:00100000:2.0:1789760790.315448:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768695066675 time 768748457365 state 0x2 ref 0 00000100:00100000:2.0:1789760790.319378:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.319985:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828608 00000100:00100000:3.0:1789760790.319996:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828608 00000100:00100000:3.0:1789760790.320049:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768695066675 time 768748457365 state 0x0 ref 1 00000100:00100000:3.0:1789760790.320064:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768748717781 time 768748457365 state 0x1 ref 1 00000100:00100000:3.0:1789760790.320090:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768748717781 time 768785379643 state 0x0 ref 1 00000100:00100000:3.0:1789760790.320102:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760790.320148:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26439680, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:3.0:1789760790.320157:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.320178:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760790.320183:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.320189:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.320203:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.354252:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973772 is committed 00002000:00010000:3.0:1789760790.354435:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26439680 -> 26443776 00010000:00010000:3.0:1789760790.354450:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.354462:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.354470:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.354477:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760790.354481:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.354502:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.354522:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.354530:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d202eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.354792:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34685us (35417us total) trans 4294973772 rc 0/0 00000100:00100000:3.0:1789760790.354802:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6484 00000100:00100000:3.0:1789760790.354812:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768748717781 time 768785379643 state 0x2 ref 0 00000100:00100000:1.0:1789760790.358562:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760790.358750:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828736 00000100:00100000:2.0:1789760790.358759:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828736 00000100:00100000:2.0:1789760790.358811:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768748717781 time 768785379643 state 0x0 ref 1 00000100:00100000:2.0:1789760790.358827:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768785640059 time 768785379643 state 0x1 ref 1 00000100:00100000:2.0:1789760790.358851:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768785640059 time 768824140697 state 0x0 ref 1 00000100:00100000:2.0:1789760790.358863:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515828736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760790.358914:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26443776, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760790.358923:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358939:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760790.358944:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.358951:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.358966:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.390129:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973773 is committed 00002000:00010000:2.0:1789760790.390209:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26443776 -> 26447872 00010000:00010000:2.0:1789760790.390224:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.390236:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.390244:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.390252:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.390256:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.390338:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.390358:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.390366:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d202f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.390499:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515828736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31624us (31942us total) trans 4294973773 rc 0/0 00000100:00100000:2.0:1789760790.390509:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6485 00000100:00100000:2.0:1789760790.390519:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768785640059 time 768824140697 state 0x2 ref 0 00000100:00100000:0.0:1789760790.395360:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760790.395454:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828864 00000100:00100000:2.0:1789760790.395463:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828864 00000100:00100000:2.0:1789760790.395689:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768785640059 time 768824140697 state 0x0 ref 1 00000100:00100000:2.0:1789760790.395703:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768824401113 time 768824140697 state 0x1 ref 1 00000100:00100000:2.0:1789760790.395834:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768824401113 time 768861123068 state 0x0 ref 1 00000100:00100000:2.0:1789760790.395848:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515828864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760790.395897:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26447872, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760790.395905:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.395920:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760790.395924:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.395929:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.395940:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.424292:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26447872 -> 26451968 00010000:00010000:2.0:1789760790.424312:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.424325:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.424335:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.424341:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.424346:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.424355:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.424375:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.424383:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d202f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.424472:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515828864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28621us (29118us total) trans 4294973774 rc 0/0 00000100:00100000:2.0:1789760790.424483:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6486 00000100:00100000:2.0:1789760790.424505:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768824401113 time 768861123068 state 0x2 ref 0 00000001:00080000:2.0:1789760790.424659:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973774 is committed 00000100:00100000:2.0:1789760790.428359:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.428572:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515828992 00000100:00100000:0.0:1789760790.428582:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515828992 00000100:00100000:0.0:1789760790.428646:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768824401113 time 768861123068 state 0x0 ref 1 00000100:00100000:0.0:1789760790.428662:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768861383484 time 768861123068 state 0x1 ref 1 00000100:00100000:0.0:1789760790.428694:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768861383484 time 768893982296 state 0x0 ref 1 00000100:00100000:0.0:1789760790.428728:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.428776:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26451968, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760790.428784:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.428798:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760790.428803:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.428809:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.428822:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.457831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973775 is committed 00002000:00010000:0.0:1789760790.457929:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26451968 -> 26456064 00010000:00010000:0.0:1789760790.457945:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.457958:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.457968:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.457976:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760790.457981:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.457991:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.458010:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.458017:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20300 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.458144:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515828992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29434us (29789us total) trans 4294973775 rc 0/0 00000100:00100000:0.0:1789760790.458155:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6487 00000100:00100000:0.0:1789760790.458164:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768861383484 time 768893982296 state 0x2 ref 0 00000100:00100000:2.0:1789760790.462991:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760790.463092:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829120 00000100:00100000:1.0:1789760790.463102:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829120 00000100:00100000:1.0:1789760790.463154:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768861383484 time 768893982296 state 0x0 ref 1 00000100:00100000:1.0:1789760790.463169:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768894242712 time 768893982296 state 0x1 ref 1 00000100:00100000:1.0:1789760790.463235:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768894242712 time 768928523641 state 0x0 ref 1 00000100:00100000:1.0:1789760790.463247:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760790.463300:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26456064, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:1.0:1789760790.463309:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.463325:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760790.463330:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.463336:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.463350:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.498812:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973776 is committed 00002000:00010000:1.0:1789760790.498951:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26456064 -> 26460160 00010000:00010000:1.0:1789760790.498968:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.498982:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.498991:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.498998:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.499004:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.499013:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.499033:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.499045:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20307 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.499173:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35923us (36188us total) trans 4294973776 rc 0/0 00000100:00100000:1.0:1789760790.499183:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6488 00000100:00100000:1.0:1789760790.499192:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768894242712 time 768928523641 state 0x2 ref 0 00000100:00100000:2.0:1789760790.503002:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.503104:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829248 00000100:00100000:0.0:1789760790.503114:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829248 00000100:00100000:0.0:1789760790.503167:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768894242712 time 768928523641 state 0x0 ref 1 00000100:00100000:0.0:1789760790.503181:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768928784057 time 768928523641 state 0x1 ref 1 00000100:00100000:0.0:1789760790.503249:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768928784057 time 768968536804 state 0x0 ref 1 00000100:00100000:0.0:1789760790.503276:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515829248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.503326:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26460160, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760790.503334:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.503349:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760790.503354:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.503360:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.503373:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.534372:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973777 is committed 00002000:00010000:1.0:1789760790.536300:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26460160 -> 26464256 00010000:00010000:1.0:1789760790.536321:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.536333:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.536342:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.536350:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.536354:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.536364:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.536391:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.536399:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2030e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.536784:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515829248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33514us (33784us total) trans 4294973777 rc 0/0 00000100:00100000:1.0:1789760790.536796:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6489 00000100:00100000:1.0:1789760790.536806:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768928784057 time 768968536804 state 0x2 ref 0 00000100:00100000:2.0:1789760790.545098:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.545302:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829376 00000100:00100000:3.0:1789760790.545313:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829376 00000100:00100000:3.0:1789760790.545366:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768928784057 time 768968536804 state 0x0 ref 1 00000100:00100000:3.0:1789760790.545381:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768968797220 time 768968536804 state 0x1 ref 1 00000100:00100000:3.0:1789760790.545412:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768968797220 time 769010700685 state 0x0 ref 1 00000100:00100000:3.0:1789760790.545424:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760790.545474:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26464256, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:3.0:1789760790.546116:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.546134:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760790.546139:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.546145:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.546164:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.573320:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26464256 -> 26468352 00010000:00010000:1.0:1789760790.573340:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.573353:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.573362:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.573369:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.573375:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.573389:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.573409:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.573418:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20315 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.573522:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515829376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28094us (28428us total) trans 4294973778 rc 0/0 00000100:00100000:1.0:1789760790.573533:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6490 00000100:00100000:1.0:1789760790.573552:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768968797220 time 769010700685 state 0x2 ref 0 00000001:00080000:2.0:1789760790.574196:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973778 is committed 00000100:00100000:2.0:1789760790.580105:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.580200:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829504 00000100:00100000:3.0:1789760790.580209:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829504 00000100:00100000:3.0:1789760790.580261:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 768968797220 time 769010700685 state 0x0 ref 1 00000100:00100000:3.0:1789760790.580277:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769010961101 time 769010700685 state 0x1 ref 1 00000100:00100000:3.0:1789760790.580307:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769010961101 time 769045596124 state 0x0 ref 1 00000100:00100000:3.0:1789760790.580318:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760790.580364:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26468352, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:3.0:1789760790.580372:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.580387:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760790.580392:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.580398:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.580412:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.620638:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973779 is committed 00002000:00010000:1.0:1789760790.622881:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26468352 -> 26472448 00010000:00010000:1.0:1789760790.622901:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.622914:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.622923:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.622930:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.622935:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.622945:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.622963:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.622971:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d2031c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.623067:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42745us (42967us total) trans 4294973779 rc 0/0 00000100:00100000:1.0:1789760790.623076:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6491 00000100:00100000:1.0:1789760790.623085:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769010961101 time 769045596124 state 0x2 ref 0 00000100:00100000:3.0:1789760790.627123:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760790.627184:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829632 00000100:00100000:2.0:1789760790.627194:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829632 00000100:00100000:2.0:1789760790.627245:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769010961101 time 769045596124 state 0x0 ref 1 00000100:00100000:2.0:1789760790.627324:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769045856540 time 769045596124 state 0x1 ref 1 00000100:00100000:2.0:1789760790.627392:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769045856540 time 769092680946 state 0x0 ref 1 00000100:00100000:2.0:1789760790.627406:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515829632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760790.627452:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26472448, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:2.0:1789760790.627461:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.627482:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760790.627486:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.627493:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.627508:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.664593:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973780 is committed 00002000:00010000:2.0:1789760790.664674:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26472448 -> 26476544 00010000:00010000:2.0:1789760790.664689:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.664702:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.664731:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.664739:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.664743:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.664753:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.664774:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.664782:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20323 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.664863:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515829632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37455us (37742us total) trans 4294973780 rc 0/0 00000100:00100000:2.0:1789760790.664874:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6492 00000100:00100000:2.0:1789760790.664883:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769045856540 time 769092680946 state 0x2 ref 0 00000100:00100000:3.0:1789760790.670447:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.670626:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829760 00000100:00100000:0.0:1789760790.670636:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829760 00000100:00100000:0.0:1789760790.670771:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769045856540 time 769092680946 state 0x0 ref 1 00000100:00100000:0.0:1789760790.670785:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769092941362 time 769092680946 state 0x1 ref 1 00000100:00100000:0.0:1789760790.670855:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769092941362 time 769136143092 state 0x0 ref 1 00000100:00100000:0.0:1789760790.670868:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.670913:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26476544, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:0.0:1789760790.670923:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.670945:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.670949:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.670955:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.670970:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.706334:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973781 is committed 00002000:00010000:0.0:1789760790.706511:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26476544 -> 26480640 00010000:00010000:0.0:1789760790.706527:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.706602:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.706611:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.706627:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760790.706631:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.706641:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.706791:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.706801:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2032a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.706932:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515829760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36061us (36487us total) trans 4294973781 rc 0/0 00000100:00100000:0.0:1789760790.706942:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6493 00000100:00100000:0.0:1789760790.706951:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769092941362 time 769136143092 state 0x2 ref 0 00000100:00100000:2.0:1789760790.714937:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760790.715323:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515829888 00000100:00100000:1.0:1789760790.715334:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515829888 00000100:00100000:1.0:1789760790.715386:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769092941362 time 769136143092 state 0x0 ref 1 00000100:00100000:1.0:1789760790.715401:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769136403508 time 769136143092 state 0x1 ref 1 00000100:00100000:1.0:1789760790.715477:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769136403508 time 769180765531 state 0x0 ref 1 00000100:00100000:1.0:1789760790.715491:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515829888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760790.715548:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26480640, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:1.0:1789760790.715558:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.715574:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.715579:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.715585:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.715598:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.744171:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26480640 -> 26484736 00010000:00010000:1.0:1789760790.744189:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.744203:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.744212:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.744219:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760790.744224:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.744234:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.744258:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.744266:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20331 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.744351:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515829888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28860us (29418us total) trans 4294973782 rc 0/0 00000100:00100000:1.0:1789760790.744360:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6494 00000100:00100000:1.0:1789760790.744369:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769136403508 time 769180765531 state 0x2 ref 0 00000001:00080000:3.0:1789760790.744477:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973782 is committed 00000100:00100000:3.0:1789760790.750594:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760790.750769:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830016 00000100:00100000:1.0:1789760790.750779:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830016 00000100:00100000:1.0:1789760790.750835:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769136403508 time 769180765531 state 0x0 ref 1 00000100:00100000:1.0:1789760790.750851:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769181025947 time 769180765531 state 0x1 ref 1 00000100:00100000:1.0:1789760790.750874:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769181025947 time 769216164046 state 0x0 ref 1 00000100:00100000:1.0:1789760790.750885:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760790.750951:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26484736, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:1.0:1789760790.750960:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.750976:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.750981:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.750988:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.751012:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.797945:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973783 is committed 00002000:00010000:2.0:1789760790.800320:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26484736 -> 26488832 00010000:00010000:2.0:1789760790.800342:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.800356:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.800365:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.800373:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.800377:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.800387:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.800406:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.800414:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20338 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.800522:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49633us (49933us total) trans 4294973783 rc 0/0 00000100:00100000:2.0:1789760790.800754:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6495 00000100:00100000:2.0:1789760790.800777:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769181025947 time 769216164046 state 0x2 ref 0 00000100:00100000:2.0:1789760790.807798:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760790.809094:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830144 00000100:00100000:1.0:1789760790.809108:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830144 00000100:00100000:1.0:1789760790.809168:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769181025947 time 769216164046 state 0x0 ref 1 00000100:00100000:1.0:1789760790.809186:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769216424462 time 769216164046 state 0x1 ref 1 00000100:00100000:1.0:1789760790.809223:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769216424462 time 769274511935 state 0x0 ref 1 00000100:00100000:1.0:1789760790.809238:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760790.809294:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26488832, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:1.0:1789760790.809303:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.809318:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.809323:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.809328:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.809352:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.848632:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760790.865009:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26488832 -> 26492928 00010000:00010000:0.0:1789760790.865029:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.865041:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.865049:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.865057:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760790.865062:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.865071:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.865096:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.865103:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d2033f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.865212:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515830144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55971us (57417us total) trans 4294973784 rc 0/0 00000100:00100000:0.0:1789760790.865222:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6496 00000100:00100000:0.0:1789760790.865231:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769216424462 time 769274511935 state 0x2 ref 0 00000001:00080000:3.0:1789760790.865453:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973784 is committed 00000100:00100000:0.0:1789760790.872210:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.872281:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830272 00000100:00100000:3.0:1789760790.872291:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830272 00000100:00100000:3.0:1789760790.872342:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769216424462 time 769274511935 state 0x0 ref 1 00000100:00100000:3.0:1789760790.872356:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769274772351 time 769274511935 state 0x1 ref 1 00000100:00100000:3.0:1789760790.872387:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769274772351 time 769337676536 state 0x0 ref 1 00000100:00100000:3.0:1789760790.872399:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760790.872447:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26492928, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd007 00010000:00010000:3.0:1789760790.872455:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.872470:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760790.872474:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.872480:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.872494:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.948774:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973785 is committed 00002000:00010000:2.0:1789760790.948932:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26492928 -> 26497024 00002000:00010000:2.0:1789760790.948941:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:2.0:1789760790.948944:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00010000:00010000:2.0:1789760790.948961:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.948975:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.948985:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.948993:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.948998:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.949009:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.949039:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.949048:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d20346 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.949179:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 76773us (76970us total) trans 4294973785 rc 0/0 00000100:00100000:2.0:1789760790.949196:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6497 00000100:00100000:2.0:1789760790.949208:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769274772351 time 769337676536 state 0x2 ref 0 00000100:00100000:2.0:1789760790.952325:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760790.952444:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830400 00000100:00100000:3.0:1789760790.952458:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830400 00000100:00100000:3.0:1789760790.952530:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769274772351 time 769337676536 state 0x0 ref 1 00000100:00100000:3.0:1789760790.952550:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769337936952 time 769337676536 state 0x1 ref 1 00000100:00100000:3.0:1789760790.952587:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769337936952 time 769417875382 state 0x0 ref 1 00000100:00100000:3.0:1789760790.952602:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760790.952680:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26497024, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:3.0:1789760790.952691:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.952747:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760790.952753:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.952762:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.952780:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.988177:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973786 is committed 00002000:00010000:2.0:1789760790.988311:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26497024 -> 26501120 00010000:00010000:2.0:1789760790.988331:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.988347:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.988357:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.988366:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760790.988372:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.988388:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.988415:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.988424:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d2034d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.988660:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36052us (36340us total) trans 4294973786 rc 0/0 00000100:00100000:2.0:1789760790.988674:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6498 00000100:00100000:2.0:1789760790.988686:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769337936952 time 769417875382 state 0x2 ref 0 00000100:00100000:2.0:1789760790.992447:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760790.993401:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830528 00000100:00100000:0.0:1789760790.993420:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830528 00000100:00100000:0.0:1789760790.993500:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769337936952 time 769417875382 state 0x0 ref 1 00000100:00100000:0.0:1789760790.993520:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769418135798 time 769417875382 state 0x1 ref 1 00000100:00100000:0.0:1789760790.993573:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769418135798 time 769458861459 state 0x0 ref 1 00000100:00100000:0.0:1789760790.993586:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760790.993662:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26501120, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:0.0:1789760790.993673:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.993694:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.993699:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.993750:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.993770:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.046587:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973787 is committed 00002000:00010000:0.0:1789760791.046780:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26501120 -> 26505216 00010000:00010000:0.0:1789760791.046800:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.046816:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.046826:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.046835:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.046841:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.046852:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.046878:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.046888:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20354 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.047021:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53430us (54582us total) trans 4294973787 rc 0/0 00000100:00100000:0.0:1789760791.047033:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6499 00000100:00100000:0.0:1789760791.047046:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769418135798 time 769458861459 state 0x2 ref 0 00000100:00100000:0.0:1789760791.050161:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760791.050420:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830656 00000100:00100000:1.0:1789760791.050434:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830656 00000100:00100000:1.0:1789760791.050505:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769418135798 time 769458861459 state 0x0 ref 1 00000100:00100000:1.0:1789760791.050524:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769459121875 time 769458861459 state 0x1 ref 1 00000100:00100000:1.0:1789760791.050558:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769459121875 time 769515846862 state 0x0 ref 1 00000100:00100000:1.0:1789760791.050573:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760791.050653:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26505216, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:1.0:1789760791.050664:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.050683:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760791.050688:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.050696:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.050745:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.097538:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973788 is committed 00002000:00010000:0.0:1789760791.097750:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26505216 -> 26509312 00010000:00010000:0.0:1789760791.097772:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.097789:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.097799:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.097808:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.097814:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.097825:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.097855:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.097863:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2035b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.097984:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515830656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47406us (47831us total) trans 4294973788 rc 0/0 00000100:00100000:0.0:1789760791.097995:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6500 00000100:00100000:0.0:1789760791.098008:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769459121875 time 769515846862 state 0x2 ref 0 00000100:00100000:1.0:1789760791.104495:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760791.105395:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830784 00000100:00100000:2.0:1789760791.105411:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830784 00000100:00100000:2.0:1789760791.105488:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769459121875 time 769515846862 state 0x0 ref 1 00000100:00100000:2.0:1789760791.105507:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769516107278 time 769515846862 state 0x1 ref 1 00000100:00100000:2.0:1789760791.105550:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769516107278 time 769570838281 state 0x0 ref 1 00000100:00100000:2.0:1789760791.105564:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760791.105640:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26509312, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:2.0:1789760791.105652:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.105669:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760791.105674:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.105681:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.105697:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.161121:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973789 is committed 00002000:00010000:0.0:1789760791.161348:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26509312 -> 26513408 00010000:00010000:0.0:1789760791.161367:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.161382:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.161392:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.161399:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.161405:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.161420:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.161457:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.161466:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d20362 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.161762:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56023us (57108us total) trans 4294973789 rc 0/0 00000100:00100000:0.0:1789760791.161777:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6501 00000100:00100000:0.0:1789760791.161790:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769516107278 time 769570838281 state 0x2 ref 0 00000100:00100000:1.0:1789760791.167046:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760791.167183:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515830912 00000100:00100000:0.0:1789760791.167197:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515830912 00000100:00100000:0.0:1789760791.167329:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769516107278 time 769570838281 state 0x0 ref 1 00000100:00100000:0.0:1789760791.167348:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769571098697 time 769570838281 state 0x1 ref 1 00000100:00100000:0.0:1789760791.167452:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769571098697 time 769632739271 state 0x0 ref 1 00000100:00100000:0.0:1789760791.167467:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515830912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760791.167594:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26513408, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:0.0:1789760791.167606:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.167634:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760791.167639:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.167648:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.167666:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.212446:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26513408 -> 26517504 00010000:00010000:1.0:1789760791.212474:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.212490:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.212498:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.212506:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760791.212511:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.212520:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.212558:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.212566:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20369 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.212764:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515830912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45241us (45674us total) trans 4294973790 rc 0/0 00000100:00100000:1.0:1789760791.212776:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6502 00000100:00100000:1.0:1789760791.212788:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769571098697 time 769632739271 state 0x2 ref 0 00000001:00080000:3.0:1789760791.214164:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973790 is committed 00000100:00100000:2.0:1789760791.217118:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760791.217216:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831040 00000100:00100000:3.0:1789760791.217228:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831040 00000100:00100000:3.0:1789760791.217313:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769571098697 time 769632739271 state 0x0 ref 1 00000100:00100000:3.0:1789760791.217333:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769632999687 time 769632739271 state 0x1 ref 1 00000100:00100000:3.0:1789760791.217410:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769632999687 time 769682698010 state 0x0 ref 1 00000100:00100000:3.0:1789760791.217424:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760791.217491:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26517504, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:3.0:1789760791.217503:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.217522:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760791.217528:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.217535:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.217562:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.291763:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973791 is committed 00002000:00010000:1.0:1789760791.292055:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26517504 -> 26521600 00010000:00010000:1.0:1789760791.292076:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.292092:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.292102:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.292111:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760791.292116:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.292125:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.292150:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.292159:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20370 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.292326:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74896us (75215us total) trans 4294973791 rc 0/0 00000100:00100000:1.0:1789760791.292340:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6503 00000100:00100000:1.0:1789760791.292352:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769632999687 time 769682698010 state 0x2 ref 0 00000100:00100000:2.0:1789760791.295242:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760791.295418:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831168 00000100:00100000:3.0:1789760791.295433:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831168 00000100:00100000:3.0:1789760791.295501:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769632999687 time 769682698010 state 0x0 ref 1 00000100:00100000:3.0:1789760791.295520:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769682958426 time 769682698010 state 0x1 ref 1 00000100:00100000:3.0:1789760791.295679:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769682958426 time 769760965822 state 0x0 ref 1 00000100:00100000:3.0:1789760791.295693:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760791.295804:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26521600, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:3.0:1789760791.295817:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.295837:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760791.295843:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.295851:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.295868:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.367416:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973792 is committed 00002000:00010000:0.0:1789760791.368934:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26521600 -> 26525696 00010000:00010000:0.0:1789760791.368959:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.368975:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.368985:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.368992:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.368997:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.369007:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.369038:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.369046:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d20377 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.369788:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 74088us (74550us total) trans 4294973792 rc 0/0 00000100:00100000:0.0:1789760791.369802:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6504 00000100:00100000:0.0:1789760791.369815:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769682958426 time 769760965822 state 0x2 ref 0 00000100:00100000:2.0:1789760791.379545:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760791.379661:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831296 00000100:00100000:0.0:1789760791.379674:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831296 00000100:00100000:0.0:1789760791.379793:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769682958426 time 769760965822 state 0x0 ref 1 00000100:00100000:0.0:1789760791.379812:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769761226238 time 769760965822 state 0x1 ref 1 00000100:00100000:0.0:1789760791.379839:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769761226238 time 769845127917 state 0x0 ref 1 00000100:00100000:0.0:1789760791.379852:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515831296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760791.379919:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26525696, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:0.0:1789760791.379930:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.379950:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760791.379956:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.379963:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.379979:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.446094:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973793 is committed 00002000:00010000:0.0:1789760791.446230:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26525696 -> 26529792 00010000:00010000:0.0:1789760791.446251:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.446265:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.446274:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.446281:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.446287:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.446296:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.446320:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.446333:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2037e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.446460:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515831296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66604us (66922us total) trans 4294973793 rc 0/0 00000100:00100000:0.0:1789760791.446472:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6505 00000100:00100000:0.0:1789760791.446492:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769761226238 time 769845127917 state 0x2 ref 0 00000100:00100000:2.0:1789760791.459064:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760791.460745:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831424 00000100:00100000:0.0:1789760791.460764:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831424 00000100:00100000:0.0:1789760791.460850:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769761226238 time 769845127917 state 0x0 ref 1 00000100:00100000:0.0:1789760791.460870:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769845388333 time 769845127917 state 0x1 ref 1 00000100:00100000:0.0:1789760791.460907:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769845388333 time 769926196078 state 0x0 ref 1 00000100:00100000:0.0:1789760791.460922:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760791.460986:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26529792, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:0.0:1789760791.460996:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.461016:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760791.461022:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.461029:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.461052:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760791.520549:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26529792 -> 26533888 00010000:00010000:0.0:1789760791.520578:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.520593:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.520603:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.520621:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.520628:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.520638:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.520663:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.520671:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20385 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.520836:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515831424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59911us (61781us total) trans 4294973794 rc 0/0 00000100:00100000:0.0:1789760791.520847:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6506 00000100:00100000:0.0:1789760791.520860:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769845388333 time 769926196078 state 0x2 ref 0 00000001:00080000:3.0:1789760791.521441:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973794 is committed 00000100:00100000:2.0:1789760791.525426:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760791.525578:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831552 00000100:00100000:0.0:1789760791.525592:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831552 00000100:00100000:0.0:1789760791.525676:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769845388333 time 769926196078 state 0x0 ref 1 00000100:00100000:0.0:1789760791.525695:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769926456494 time 769926196078 state 0x1 ref 1 00000100:00100000:0.0:1789760791.525934:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769926456494 time 769991221480 state 0x0 ref 1 00000100:00100000:0.0:1789760791.525949:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515831552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760791.526011:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26533888, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:0.0:1789760791.526021:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.526041:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760791.526046:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.526054:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.526070:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760791.582422:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26533888 -> 26537984 00010000:00010000:0.0:1789760791.582454:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.582470:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.582482:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.582490:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.582497:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.582508:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.582667:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.582677:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d2038c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.582844:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515831552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 56890us (57424us total) trans 4294973795 rc 0/0 00000100:00100000:0.0:1789760791.582862:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6507 00000100:00100000:0.0:1789760791.582876:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769926456494 time 769991221480 state 0x2 ref 0 00000001:00080000:3.0:1789760791.582999:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973795 is committed 00000100:00100000:2.0:1789760791.589475:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760791.589556:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831680 00000100:00100000:1.0:1789760791.589571:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831680 00000100:00100000:1.0:1789760791.589654:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769926456494 time 769991221480 state 0x0 ref 1 00000100:00100000:1.0:1789760791.589672:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769991481896 time 769991221480 state 0x1 ref 1 00000100:00100000:1.0:1789760791.589750:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769991481896 time 770055038335 state 0x0 ref 1 00000100:00100000:1.0:1789760791.589766:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760791.589831:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26537984, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:1.0:1789760791.589841:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.589863:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.589869:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.589877:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.589895:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.639030:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973796 is committed 00002000:00010000:0.0:1789760791.639292:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26537984 -> 26542080 00010000:00010000:0.0:1789760791.639311:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.639322:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.639330:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.639334:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.639339:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.639346:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.639369:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.639375:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20393 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.639485:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 49715us (50018us total) trans 4294973796 rc 0/0 00000100:00100000:0.0:1789760791.639497:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6508 00000100:00100000:0.0:1789760791.639509:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769991481896 time 770055038335 state 0x2 ref 0 00000100:00100000:3.0:1789760791.651086:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760791.652498:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831808 00000100:00100000:1.0:1789760791.652517:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831808 00000100:00100000:1.0:1789760791.652621:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 769991481896 time 770055038335 state 0x0 ref 1 00000100:00100000:1.0:1789760791.652644:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770055298751 time 770055038335 state 0x1 ref 1 00000100:00100000:1.0:1789760791.652684:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770055298751 time 770117973132 state 0x0 ref 1 00000100:00100000:1.0:1789760791.652700:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515831808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760791.652817:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26542080, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:1.0:1789760791.652829:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.652848:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.652852:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.652860:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.652877:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.708346:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973797 is committed 00002000:00010000:3.0:1789760791.710325:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26542080 -> 26546176 00010000:00010000:3.0:1789760791.710359:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.710374:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.710385:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.710393:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760791.710399:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.710415:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.710439:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.710446:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d2039a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.710569:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515831808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 57866us (59491us total) trans 4294973797 rc 0/0 00000100:00100000:3.0:1789760791.710582:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6509 00000100:00100000:3.0:1789760791.710595:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770055298751 time 770117973132 state 0x2 ref 0 00000100:00100000:3.0:1789760791.714300:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760791.714978:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515831936 00000100:00100000:2.0:1789760791.714993:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515831936 00000100:00100000:2.0:1789760791.715067:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770055298751 time 770117973132 state 0x0 ref 1 00000100:00100000:2.0:1789760791.715085:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770118233548 time 770117973132 state 0x1 ref 1 00000100:00100000:2.0:1789760791.715121:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770118233548 time 770180409863 state 0x0 ref 1 00000100:00100000:2.0:1789760791.715139:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760791.716604:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26546176, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:2.0:1789760791.716631:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.716653:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760791.716659:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.716667:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.716685:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.785519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973798 is committed 00002000:00010000:3.0:1789760791.785758:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26546176 -> 26550272 00010000:00010000:3.0:1789760791.785779:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.785795:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.785805:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.785818:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760791.785824:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.785834:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.785860:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.785869:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d203a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.785994:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515831936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 70849us (71700us total) trans 4294973798 rc 0/0 00000100:00100000:3.0:1789760791.786007:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6510 00000100:00100000:3.0:1789760791.786020:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770118233548 time 770180409863 state 0x2 ref 0 00000100:00100000:2.0:1789760791.793579:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760791.793698:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832064 00000100:00100000:0.0:1789760791.793749:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832064 00000100:00100000:0.0:1789760791.793826:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770118233548 time 770180409863 state 0x0 ref 1 00000100:00100000:0.0:1789760791.793843:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770180670279 time 770180409863 state 0x1 ref 1 00000100:00100000:0.0:1789760791.793870:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770180670279 time 770259160054 state 0x0 ref 1 00000100:00100000:0.0:1789760791.793883:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760791.793948:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26550272, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:0.0:1789760791.793959:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.793979:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760791.793985:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.793992:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.794008:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.869893:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760791.879464:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973799 is committed 00002000:00010000:0.0:1789760791.880269:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26550272 -> 26554368 00010000:00010000:0.0:1789760791.880290:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.880305:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.880315:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.880322:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760791.880329:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.880340:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.880364:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.880372:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d203a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.880702:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 86810us (87128us total) trans 4294973799 rc 0/0 00000100:00100000:2.0:1789760791.880733:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6511 00000100:00100000:2.0:1789760791.880747:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770180670279 time 770259160054 state 0x2 ref 0 00000100:00100000:3.0:1789760791.883478:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760791.883666:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832192 00000100:00100000:0.0:1789760791.883680:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832192 00000100:00100000:0.0:1789760791.883777:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770180670279 time 770259160054 state 0x0 ref 1 00000100:00100000:0.0:1789760791.883797:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770259420470 time 770259160054 state 0x1 ref 1 00000100:00100000:3.0:1789760791.883903:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770259420470 time 770349189457 state 0x0 ref 1 00000100:00100000:3.0:1789760791.883919:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760791.883984:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26554368, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd007 00010000:00010000:3.0:1789760791.883995:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.884014:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.884019:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.884028:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.884043:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.944512:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973800 is committed 00002000:00010000:3.0:1789760791.947588:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26554368 -> 26558464 00002000:00010000:3.0:1789760791.947601:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:3.0:1789760791.947614:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00010000:00010000:3.0:1789760791.947632:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.947648:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.947658:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.947666:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760791.947671:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.947687:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.947742:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.947753:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.947886:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 63964us (64415us total) trans 4294973800 rc 0/0 00000100:00100000:3.0:1789760791.947898:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6512 00000100:00100000:3.0:1789760791.947910:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770259420470 time 770349189457 state 0x2 ref 0 00000100:00100000:3.0:1789760791.950477:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760791.950769:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832320 00000100:00100000:2.0:1789760791.950783:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832320 00000100:00100000:2.0:1789760791.950855:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770259420470 time 770349189457 state 0x0 ref 1 00000100:00100000:2.0:1789760791.950874:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770349449873 time 770349189457 state 0x1 ref 1 00000100:00100000:2.0:1789760791.950902:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770349449873 time 770416191962 state 0x0 ref 1 00000100:00100000:2.0:1789760791.950917:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760791.950980:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26558464, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760791.950989:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.951009:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760791.951013:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.951021:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.951037:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.988038:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26558464 -> 26562560 00010000:00010000:2.0:1789760791.988065:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.988082:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.988092:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.988111:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760791.988117:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.988128:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.988155:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.988164:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d203b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.988334:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515832320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37412us (37863us total) trans 4294973801 rc 0/0 00000100:00100000:2.0:1789760791.988347:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6513 00000100:00100000:2.0:1789760791.988360:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770349449873 time 770416191962 state 0x2 ref 0 00000001:00080000:1.0:1789760791.988522:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973801 is committed 00000100:00100000:0.0:1789760791.992695:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760791.992808:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832448 00000100:00100000:1.0:1789760791.992821:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832448 00000100:00100000:1.0:1789760791.992889:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770349449873 time 770416191962 state 0x0 ref 1 00000100:00100000:1.0:1789760791.992906:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770416452378 time 770416191962 state 0x1 ref 1 00000100:00100000:1.0:1789760791.992939:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770416452378 time 770458229113 state 0x0 ref 1 00000100:00100000:1.0:1789760791.992954:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760791.993015:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26562560, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:1.0:1789760791.993026:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.993046:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.993051:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.993059:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.993077:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.026227:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973802 is committed 00002000:00010000:0.0:1789760792.026392:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26562560 -> 26566656 00010000:00010000:0.0:1789760792.026414:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.026431:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.026442:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.026451:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.026457:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.026469:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.026496:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.026505:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000916d823/0x69dc3c3d61d203bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.026801:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33842us (34110us total) trans 4294973802 rc 0/0 00000100:00100000:0.0:1789760792.026816:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6514 00000100:00100000:0.0:1789760792.026829:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770416452378 time 770458229113 state 0x2 ref 0 00000100:00100000:2.0:1789760792.030154:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.030237:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832576 00000100:00100000:3.0:1789760792.030262:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832576 00000100:00100000:3.0:1789760792.030364:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770416452378 time 770458229113 state 0x0 ref 1 00000100:00100000:3.0:1789760792.030384:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770458489529 time 770458229113 state 0x1 ref 1 00000100:00100000:3.0:1789760792.030414:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770458489529 time 770495702521 state 0x0 ref 1 00000100:00100000:3.0:1789760792.030427:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.030490:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26566656, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.030502:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.030522:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.030528:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.030548:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.030565:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760792.031461:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760792.058634:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973803 is committed 00002000:00010000:1.0:1789760792.058852:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26566656 -> 26570752 00010000:00010000:1.0:1789760792.058873:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.058887:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.058896:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.058905:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760792.058911:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.058921:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.058945:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.058952:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.059106:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28674us (28962us total) trans 4294973803 rc 0/0 00000100:00100000:1.0:1789760792.059118:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6515 00000100:00100000:1.0:1789760792.059130:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770458489529 time 770495702521 state 0x2 ref 0 00000100:00100000:0.0:1789760792.062317:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.062423:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832704 00000100:00100000:3.0:1789760792.062435:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832704 00000100:00100000:3.0:1789760792.062503:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770458489529 time 770495702521 state 0x0 ref 1 00000100:00100000:3.0:1789760792.062522:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770495962937 time 770495702521 state 0x1 ref 1 00000100:00100000:3.0:1789760792.062673:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770495962937 time 770527960312 state 0x0 ref 1 00000100:00100000:3.0:1789760792.062687:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.062783:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26570752, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.062794:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.062815:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.062820:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.062827:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.062846:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.092204:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26570752 -> 26574848 00010000:00010000:3.0:1789760792.092231:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.092249:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.092259:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.092268:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760792.092274:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.092284:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.092309:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.092318:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.092441:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515832704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29749us (30188us total) trans 4294973804 rc 0/0 00000100:00100000:3.0:1789760792.092453:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6516 00000100:00100000:3.0:1789760792.092466:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770495962937 time 770527960312 state 0x2 ref 0 00000001:00080000:1.0:1789760792.092978:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973804 is committed 00000100:00100000:2.0:1789760792.097924:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760792.098066:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832832 00000100:00100000:0.0:1789760792.098081:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832832 00000100:00100000:0.0:1789760792.098155:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770495962937 time 770527960312 state 0x0 ref 1 00000100:00100000:0.0:1789760792.098174:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770528220728 time 770527960312 state 0x1 ref 1 00000100:00100000:0.0:1789760792.098212:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770528220728 time 770563500434 state 0x0 ref 1 00000100:00100000:0.0:1789760792.098225:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760792.098302:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26574848, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:0.0:1789760792.098313:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.098333:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760792.098338:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.098345:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.098368:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.128784:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973805 is committed 00002000:00010000:2.0:1789760792.128908:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26574848 -> 26578944 00010000:00010000:2.0:1789760792.128928:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.128943:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.128953:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.128963:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760792.128969:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.128980:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.129006:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.129015:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.129136:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8063b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515832832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30905us (31219us total) trans 4294973805 rc 0/0 00000100:00100000:2.0:1789760792.129148:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6517 00000100:00100000:2.0:1789760792.129159:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770528220728 time 770563500434 state 0x2 ref 0 00000100:00100000:2.0:1789760792.132653:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.132793:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515832960 00000100:00100000:3.0:1789760792.132807:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515832960 00000100:00100000:3.0:1789760792.132879:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770528220728 time 770563500434 state 0x0 ref 1 00000100:00100000:3.0:1789760792.132898:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770563760850 time 770563500434 state 0x1 ref 1 00000100:00100000:3.0:1789760792.132969:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770563760850 time 770598256238 state 0x0 ref 1 00000100:00100000:3.0:1789760792.132984:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.133049:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26578944, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.133060:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.133081:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.133086:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.133094:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.133110:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.168165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973806 is committed 00002000:00010000:0.0:1789760792.168355:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26578944 -> 26583040 00010000:00010000:0.0:1789760792.168375:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.168389:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.168399:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.168407:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.168412:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.168421:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.168445:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.168453:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d203d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.168662:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515832960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35671us (36016us total) trans 4294973806 rc 0/0 00000100:00100000:0.0:1789760792.168675:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6518 00000100:00100000:0.0:1789760792.168687:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770563760850 time 770598256238 state 0x2 ref 0 00000100:00100000:0.0:1789760792.171643:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760792.171824:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833088 00000100:00100000:2.0:1789760792.171838:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833088 00000100:00100000:2.0:1789760792.171908:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770563760850 time 770598256238 state 0x0 ref 1 00000100:00100000:2.0:1789760792.171928:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770598516654 time 770598256238 state 0x1 ref 1 00000100:00100000:2.0:1789760792.171962:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770598516654 time 770637251324 state 0x0 ref 1 00000100:00100000:2.0:1789760792.171976:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515833088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.172036:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26583040, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.172047:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.172066:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760792.172072:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.172079:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.172095:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.208491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973807 is committed 00002000:00010000:0.0:1789760792.208682:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26583040 -> 26587136 00010000:00010000:0.0:1789760792.208703:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.208737:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.208749:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.208758:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.208763:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.208774:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.208797:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.208812:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d203e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.208931:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515833088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36948us (37294us total) trans 4294973807 rc 0/0 00000100:00100000:0.0:1789760792.208942:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6519 00000100:00100000:0.0:1789760792.208954:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770598516654 time 770637251324 state 0x2 ref 0 00000100:00100000:1.0:1789760792.212313:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760792.213139:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833216 00000100:00100000:0.0:1789760792.213155:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833216 00000100:00100000:0.0:1789760792.213236:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770598516654 time 770637251324 state 0x0 ref 1 00000100:00100000:0.0:1789760792.213322:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770637511740 time 770637251324 state 0x1 ref 1 00000100:00100000:0.0:1789760792.213456:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770637511740 time 770678743325 state 0x0 ref 1 00000100:00100000:0.0:1789760792.213472:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760792.213551:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26587136, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:0.0:1789760792.213562:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.213581:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760792.213586:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.213594:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.213626:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.249171:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973808 is committed 00002000:00010000:0.0:1789760792.249344:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26587136 -> 26591232 00010000:00010000:0.0:1789760792.249366:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.249383:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.249393:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.249402:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.249408:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.249418:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.249444:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.249454:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d203e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.249670:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36194us (37363us total) trans 4294973808 rc 0/0 00000100:00100000:0.0:1789760792.249683:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6520 00000100:00100000:0.0:1789760792.249696:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770637511740 time 770678743325 state 0x2 ref 0 00000100:00100000:1.0:1789760792.253963:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.254084:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833344 00000100:00100000:3.0:1789760792.254098:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833344 00000100:00100000:3.0:1789760792.254170:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770637511740 time 770678743325 state 0x0 ref 1 00000100:00100000:3.0:1789760792.254191:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770679003741 time 770678743325 state 0x1 ref 1 00000100:00100000:3.0:1789760792.254269:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770679003741 time 770719556511 state 0x0 ref 1 00000100:00100000:3.0:1789760792.254284:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.254347:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26591232, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.254357:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.254376:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.254381:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.254389:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.254404:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.282672:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973809 is committed 00002000:00010000:3.0:1789760792.282802:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26591232 -> 26595328 00010000:00010000:3.0:1789760792.282821:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.282838:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.282849:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.282858:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760792.282864:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.282876:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.282903:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.282912:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d203ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.283019:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8056a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28731us (29066us total) trans 4294973809 rc 0/0 00000100:00100000:3.0:1789760792.283030:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6521 00000100:00100000:3.0:1789760792.283041:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770679003741 time 770719556511 state 0x2 ref 0 00000100:00100000:1.0:1789760792.285818:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.285983:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833472 00000100:00100000:3.0:1789760792.285996:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833472 00000100:00100000:3.0:1789760792.286062:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770679003741 time 770719556511 state 0x0 ref 1 00000100:00100000:3.0:1789760792.286082:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770719816927 time 770719556511 state 0x1 ref 1 00000100:00100000:3.0:1789760792.286152:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770719816927 time 770751439923 state 0x0 ref 1 00000100:00100000:3.0:1789760792.286166:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.286233:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26595328, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.286254:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.286274:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.286280:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.286287:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.286303:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.322056:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26595328 -> 26599424 00010000:00010000:2.0:1789760792.322082:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.322097:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.322106:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.322114:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760792.322121:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.322131:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.322156:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.322164:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009bd1595a/0x69dc3c3d61d203f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.322294:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515833472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36123us (36483us total) trans 4294973810 rc 0/0 00000100:00100000:2.0:1789760792.322307:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6522 00000100:00100000:2.0:1789760792.322318:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770719816927 time 770751439923 state 0x2 ref 0 00000001:00080000:2.0:1789760792.325200:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973810 is committed 00000100:00100000:1.0:1789760792.327972:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760792.328084:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833600 00000100:00100000:2.0:1789760792.328097:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833600 00000100:00100000:2.0:1789760792.328176:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770719816927 time 770751439923 state 0x0 ref 1 00000100:00100000:2.0:1789760792.328198:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770751700339 time 770751439923 state 0x1 ref 1 00000100:00100000:2.0:1789760792.328345:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770751700339 time 770793631707 state 0x0 ref 1 00000100:00100000:2.0:1789760792.328360:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515833600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.328432:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26599424, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.328446:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.328469:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760792.328474:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.328482:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.328501:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.363819:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26599424 -> 26603520 00010000:00010000:2.0:1789760792.363844:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.363860:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.363871:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.363879:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760792.363885:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.363896:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.363926:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.363935:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d203fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.364051:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515833600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35686us (36088us total) trans 4294973811 rc 0/0 00000100:00100000:2.0:1789760792.364063:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6523 00000100:00100000:2.0:1789760792.364075:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770751700339 time 770793631707 state 0x2 ref 0 00000001:00080000:1.0:1789760792.366977:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973811 is committed 00000100:00100000:0.0:1789760792.368479:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.368663:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833728 00000100:00100000:3.0:1789760792.368677:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833728 00000100:00100000:3.0:1789760792.368860:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770751700339 time 770793631707 state 0x0 ref 1 00000100:00100000:3.0:1789760792.368879:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770793892123 time 770793631707 state 0x1 ref 1 00000100:00100000:2.0:1789760792.369003:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770793892123 time 770834289327 state 0x0 ref 1 00000100:00100000:2.0:1789760792.369188:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.369316:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26603520, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.369329:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.369349:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760792.369355:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.369363:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.369379:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.398476:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973812 is committed 00002000:00010000:3.0:1789760792.398682:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26603520 -> 26607616 00010000:00010000:3.0:1789760792.398701:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.398752:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.398764:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.398773:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760792.398779:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.398790:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.398814:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.398823:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d20403 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.398947:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515833728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29924us (30472us total) trans 4294973812 rc 0/0 00000100:00100000:3.0:1789760792.398960:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6524 00000100:00100000:3.0:1789760792.398973:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770793892123 time 770834289327 state 0x2 ref 0 00000100:00100000:1.0:1789760792.401649:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760792.401797:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833856 00000100:00100000:2.0:1789760792.401810:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833856 00000100:00100000:2.0:1789760792.401880:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770793892123 time 770834289327 state 0x0 ref 1 00000100:00100000:2.0:1789760792.401898:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770834549743 time 770834289327 state 0x1 ref 1 00000100:00100000:2.0:1789760792.401928:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770834549743 time 770867217383 state 0x0 ref 1 00000100:00100000:2.0:1789760792.401942:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515833856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.402005:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26607616, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.402016:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.402036:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760792.402042:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.402049:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.402067:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.432412:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973813 is committed 00002000:00010000:0.0:1789760792.432596:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26607616 -> 26611712 00010000:00010000:0.0:1789760792.432627:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.432644:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.432656:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.432665:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.432670:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.432680:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.432704:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.432742:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2040a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.432897:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515833856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30949us (31255us total) trans 4294973813 rc 0/0 00000100:00100000:0.0:1789760792.432910:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6525 00000100:00100000:0.0:1789760792.432923:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770834549743 time 770867217383 state 0x2 ref 0 00000100:00100000:1.0:1789760792.436439:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760792.436616:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515833984 00000100:00100000:0.0:1789760792.436629:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515833984 00000100:00100000:0.0:1789760792.436703:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770834549743 time 770867217383 state 0x0 ref 1 00000100:00100000:0.0:1789760792.436752:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770867477799 time 770867217383 state 0x1 ref 1 00000100:00100000:0.0:1789760792.436830:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770867477799 time 770902118653 state 0x0 ref 1 00000100:00100000:0.0:1789760792.436846:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515833984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760792.436914:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26611712, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:0.0:1789760792.436925:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.436944:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760792.436950:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.436958:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.436980:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.467408:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973814 is committed 00002000:00010000:0.0:1789760792.467624:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26611712 -> 26615808 00010000:00010000:0.0:1789760792.467645:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.467662:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.467673:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.467681:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.467687:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.467697:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.467803:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.467812:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20411 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.467989:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515833984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31140us (31558us total) trans 4294973814 rc 0/0 00000100:00100000:0.0:1789760792.468003:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6526 00000100:00100000:0.0:1789760792.468015:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770867477799 time 770902118653 state 0x2 ref 0 00000100:00100000:0.0:1789760792.472003:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.472119:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834112 00000100:00100000:3.0:1789760792.472135:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834112 00000100:00100000:3.0:1789760792.472206:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770867477799 time 770902118653 state 0x0 ref 1 00000100:00100000:3.0:1789760792.472224:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770902379069 time 770902118653 state 0x1 ref 1 00000100:00100000:3.0:1789760792.472353:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770902379069 time 770937640400 state 0x0 ref 1 00000100:00100000:3.0:1789760792.472367:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515834112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.472433:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26615808, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.472444:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.472464:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.472469:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.472476:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.472493:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.503780:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973815 is committed 00002000:00010000:0.0:1789760792.504055:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26615808 -> 26619904 00010000:00010000:0.0:1789760792.504077:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.504092:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.504101:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.504110:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36cc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.504116:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.504126:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.504152:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.504166:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daf8a260/0x69dc3c3d61d20418 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.504307:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515834112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31934us (32311us total) trans 4294973815 rc 0/0 00000100:00100000:0.0:1789760792.504320:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6527 00000100:00100000:0.0:1789760792.504335:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770902379069 time 770937640400 state 0x2 ref 0 00000100:00100000:1.0:1789760792.508141:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.508309:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834240 00000100:00100000:3.0:1789760792.508322:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834240 00000100:00100000:3.0:1789760792.508395:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770902379069 time 770937640400 state 0x0 ref 1 00000100:00100000:3.0:1789760792.508415:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770937900816 time 770937640400 state 0x1 ref 1 00000100:00100000:3.0:1789760792.508488:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770937900816 time 770973776335 state 0x0 ref 1 00000100:00100000:3.0:1789760792.508507:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515834240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.508585:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26619904, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.508596:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.508627:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760792.508632:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.508640:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.508656:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.536168:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26619904 -> 26624000 00010000:00010000:3.0:1789760792.536197:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.536214:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.536223:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.536231:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760792.536237:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.536319:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.536353:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.536361:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d2041f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.536498:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515834240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27985us (28364us total) trans 4294973816 rc 0/0 00000100:00100000:3.0:1789760792.536513:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6528 00000100:00100000:3.0:1789760792.538046:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770937900816 time 770973776335 state 0x2 ref 0 00000001:00080000:1.0:1789760792.538308:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973816 is committed 00000100:00100000:1.0:1789760792.540028:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760792.540397:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834368 00000100:00100000:2.0:1789760792.540411:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834368 00000100:00100000:2.0:1789760792.540482:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770937900816 time 770973776335 state 0x0 ref 1 00000100:00100000:2.0:1789760792.540500:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770974036751 time 770973776335 state 0x1 ref 1 00000100:00100000:2.0:1789760792.540639:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770974036751 time 771005926113 state 0x0 ref 1 00000100:00100000:2.0:1789760792.540655:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515834368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.540749:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26624000, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.540761:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.540780:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760792.540785:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.540793:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.540809:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.588489:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973817 is committed 00002000:00010000:3.0:1789760792.588580:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26624000 -> 26628096 00010000:00010000:3.0:1789760792.588597:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.588623:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.588632:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.588639:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760792.588645:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.588654:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.588673:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.588681:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20426 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.588809:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515834368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48151us (48786us total) trans 4294973817 rc 0/0 00000100:00100000:3.0:1789760792.588820:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6529 00000100:00100000:3.0:1789760792.588830:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770974036751 time 771005926113 state 0x2 ref 0 00000100:00100000:2.0:1789760792.600223:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.600568:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834496 00000100:00100000:3.0:1789760792.600579:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834496 00000100:00100000:3.0:1789760792.600642:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 770974036751 time 771005926113 state 0x0 ref 1 00000100:00100000:3.0:1789760792.600658:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771006186529 time 771005926113 state 0x1 ref 1 00000100:00100000:3.0:1789760792.600691:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771006186529 time 771065979841 state 0x0 ref 1 00000100:00100000:3.0:1789760792.600704:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515834496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.600794:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26628096, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.600803:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.600818:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.600823:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.600829:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.600843:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.643765:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26628096 -> 26632192 00010000:00010000:2.0:1789760792.643787:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.643801:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.643810:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.643824:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36de00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760792.643829:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.643838:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.643855:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.643863:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c67abbd2/0x69dc3c3d61d2042d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.643957:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515834496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43251us (43738us total) trans 4294973818 rc 0/0 00000100:00100000:2.0:1789760792.643967:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6530 00000100:00100000:2.0:1789760792.643977:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771006186529 time 771065979841 state 0x2 ref 0 00000001:00080000:1.0:1789760792.645417:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973818 is committed 00000100:00100000:3.0:1789760792.647372:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760792.647533:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834624 00000100:00100000:0.0:1789760792.647544:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834624 00000100:00100000:0.0:1789760792.647608:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771006186529 time 771065979841 state 0x0 ref 1 00000100:00100000:0.0:1789760792.647626:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771066240257 time 771065979841 state 0x1 ref 1 00000100:00100000:0.0:1789760792.647653:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771066240257 time 771112942498 state 0x0 ref 1 00000100:00100000:0.0:1789760792.647666:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515834624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760792.647754:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26632192, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:0.0:1789760792.647763:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.647779:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.647784:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.647790:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.647808:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.684141:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973819 is committed 00002000:00010000:0.0:1789760792.684236:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26632192 -> 26636288 00010000:00010000:0.0:1789760792.684263:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.684278:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.684287:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.684295:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.684300:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.684309:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.684328:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.684467:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20434 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.684586:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515834624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36917us (37217us total) trans 4294973819 rc 0/0 00000100:00100000:0.0:1789760792.684596:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6531 00000100:00100000:0.0:1789760792.684614:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771066240257 time 771112942498 state 0x2 ref 0 00000100:00100000:0.0:1789760792.689578:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760792.689842:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834752 00000100:00100000:2.0:1789760792.689852:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834752 00000100:00100000:2.0:1789760792.689904:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771066240257 time 771112942498 state 0x0 ref 1 00000100:00100000:2.0:1789760792.689919:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771113202914 time 771112942498 state 0x1 ref 1 00000100:00100000:2.0:1789760792.690040:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771113202914 time 771155327989 state 0x0 ref 1 00000100:00100000:2.0:1789760792.690054:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515834752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.690108:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26636288, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.690118:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.690134:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760792.690139:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.690145:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.690160:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.734015:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973820 is committed 00002000:00010000:0.0:1789760792.734190:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26636288 -> 26640384 00010000:00010000:0.0:1789760792.734205:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.734218:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.734227:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.734235:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.734239:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.734321:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.734409:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.734423:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d2043b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.734556:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515834752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44499us (44981us total) trans 4294973820 rc 0/0 00000100:00100000:0.0:1789760792.734566:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6532 00000100:00100000:0.0:1789760792.734575:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771113202914 time 771155327989 state 0x2 ref 0 00000100:00100000:2.0:1789760792.738206:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760792.738576:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515834880 00000100:00100000:0.0:1789760792.738586:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515834880 00000100:00100000:0.0:1789760792.741463:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771113202914 time 771155327989 state 0x0 ref 1 00000100:00100000:0.0:1789760792.741488:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771155588405 time 771155327989 state 0x1 ref 1 00000100:00100000:2.0:1789760792.741763:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771155588405 time 771207049692 state 0x0 ref 1 00000100:00100000:2.0:1789760792.741790:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515834880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760792.741849:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26640384, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:2.0:1789760792.741858:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.741876:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760792.741881:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.741887:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.741902:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.770792:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973821 is committed 00002000:00010000:2.0:1789760792.770957:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26640384 -> 26644480 00010000:00010000:2.0:1789760792.770973:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.770986:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.770996:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.771004:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760792.771009:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.771018:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.771038:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.771046:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20442 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.771142:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515834880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 29360us (32940us total) trans 4294973821 rc 0/0 00000100:00100000:2.0:1789760792.771154:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6533 00000100:00100000:2.0:1789760792.771163:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771155588405 time 771207049692 state 0x2 ref 0 00000100:00100000:3.0:1789760792.773614:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760792.774118:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835008 00000100:00100000:2.0:1789760792.774130:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835008 00000100:00100000:3.0:1789760792.774376:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771155588405 time 771207049692 state 0x0 ref 1 00000100:00100000:3.0:1789760792.774392:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771207310108 time 771207049692 state 0x1 ref 1 00000100:00100000:3.0:1789760792.774422:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771207310108 time 771239710863 state 0x0 ref 1 00000100:00100000:3.0:1789760792.774435:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515835008:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.774479:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26644480, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.774488:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.774503:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.774507:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.774514:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.774651:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.812555:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973822 is committed 00002000:00010000:2.0:1789760792.812734:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26644480 -> 26648576 00010000:00010000:2.0:1789760792.812750:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.812763:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.812772:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.812780:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ca00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760792.812785:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.812794:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.812813:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.812824:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afa1270d/0x69dc3c3d61d20449 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.813049:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515835008:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38611us (39447us total) trans 4294973822 rc 0/0 00000100:00100000:2.0:1789760792.813060:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6534 00000100:00100000:2.0:1789760792.813070:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771207310108 time 771239710863 state 0x2 ref 0 00000100:00100000:0.0:1789760792.818286:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.822104:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835136 00000100:00100000:3.0:1789760792.822118:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835136 00000100:00100000:3.0:1789760792.822181:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771207310108 time 771239710863 state 0x0 ref 1 00000100:00100000:3.0:1789760792.822196:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771239971279 time 771239710863 state 0x1 ref 1 00000100:00100000:3.0:1789760792.822340:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771239971279 time 771287626714 state 0x0 ref 1 00000100:00100000:3.0:1789760792.822354:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835136:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.822405:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26648576, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd007 00010000:00010000:3.0:1789760792.822414:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.822430:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.822434:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.822439:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.822453:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.872831:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973823 is committed 00002000:00010000:0.0:1789760792.872973:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26648576 -> 26652672 00002000:00010000:0.0:1789760792.872978:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:0.0:1789760792.872981:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00010000:00010000:0.0:1789760792.872994:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.873007:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.873017:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.873026:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.873031:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.873039:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.873058:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.873065:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e084edc7/0x69dc3c3d61d20450 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.873155:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835136:12345-192.168.203.46@tcp:4:dd.0 Request processed in 50799us (54875us total) trans 4294973823 rc 0/0 00000100:00100000:0.0:1789760792.873165:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6535 00000100:00100000:0.0:1789760792.873175:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771239971279 time 771287626714 state 0x2 ref 0 00000100:00100000:0.0:1789760792.879820:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760792.879952:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835264 00000100:00100000:3.0:1789760792.879963:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835264 00000100:00100000:3.0:1789760792.880015:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771239971279 time 771287626714 state 0x0 ref 1 00000100:00100000:3.0:1789760792.880030:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771287887130 time 771287626714 state 0x1 ref 1 00000100:00100000:3.0:1789760792.880061:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771287887130 time 771345350178 state 0x0 ref 1 00000100:00100000:3.0:1789760792.880072:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835264:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760792.880119:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26652672, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:3.0:1789760792.880127:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.880142:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.880147:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.880153:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.880167:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.893846:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760792.954021:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973824 is committed 00002000:00010000:0.0:1789760792.956079:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26652672 -> 26656768 00010000:00010000:0.0:1789760792.956097:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.956110:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.956120:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.956127:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760792.956132:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.956140:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.956159:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.956166:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee664292/0x69dc3c3d61d20457 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.956274:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835264:12345-192.168.203.46@tcp:4:dd.0 Request processed in 76199us (76458us total) trans 4294973824 rc 0/0 00000100:00100000:0.0:1789760792.956285:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6536 00000100:00100000:0.0:1789760792.956294:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771287887130 time 771345350178 state 0x2 ref 0 00000100:00100000:1.0:1789760792.961040:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760792.961124:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835392 00000100:00100000:0.0:1789760792.961135:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835392 00000100:00100000:0.0:1789760792.961186:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771287887130 time 771345350178 state 0x0 ref 1 00000100:00100000:0.0:1789760792.961200:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771345610594 time 771345350178 state 0x1 ref 1 00000100:00100000:0.0:1789760792.961228:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771345610594 time 771426517331 state 0x0 ref 1 00000100:00100000:0.0:1789760792.961239:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760792.961285:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26656768, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:0.0:1789760792.961295:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961311:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.961316:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.961322:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961338:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.041925:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973825 is committed 00002000:00010000:0.0:1789760793.042035:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26656768 -> 26660864 00010000:00010000:0.0:1789760793.042052:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.042065:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.042073:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.042081:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.042086:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.042095:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.042114:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.042122:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2045e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.042208:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835392:12345-192.168.203.46@tcp:4:dd.0 Request processed in 80966us (81173us total) trans 4294973825 rc 0/0 00000100:00100000:0.0:1789760793.042217:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6537 00000100:00100000:0.0:1789760793.042227:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771345610594 time 771426517331 state 0x2 ref 0 00000100:00100000:1.0:1789760793.060993:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760793.061073:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835520 00000100:00100000:0.0:1789760793.061084:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835520 00000100:00100000:0.0:1789760793.061135:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771345610594 time 771426517331 state 0x0 ref 1 00000100:00100000:0.0:1789760793.061151:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771426777747 time 771426517331 state 0x1 ref 1 00000100:00100000:0.0:1789760793.061184:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771426777747 time 771526472982 state 0x0 ref 1 00000100:00100000:0.0:1789760793.061196:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835520:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760793.061247:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26660864, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:0.0:1789760793.061256:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.061271:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760793.061275:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.061282:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.061296:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.086050:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e34a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544422528:0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.086118:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760793.086176:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789760793.086183:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544422144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.086211:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760793.086233:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544422656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.086260:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760793.086291:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422656 00000100:00100000:3.0:1789760793.086298:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422656 00000100:00100000:3.0:1789760793.086348:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1789760793.086357:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544422656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.086406:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544422656:12345-0@lo:400:kworker.0 Request processed in 50us (144us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.086413:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1789760793.086454:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544422656:0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.086760:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544422400:0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.086792:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760793.086822:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760793.086828:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae804e36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544422272:0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.086850:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760793.086869:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae806191880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544422784:0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.086889:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760793.086915:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422400 00000100:00100000:2.0:1789760793.086919:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422400 00000100:00100000:2.0:1789760793.086942:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422272 00000100:00100000:2.0:1789760793.086946:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422272 00000100:00100000:2.0:1789760793.086962:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:2.0:1789760793.086969:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544422400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.087006:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544422400:12345-0@lo:400:kworker.0 Request processed in 37us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760793.087027:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:2.0:1789760793.087041:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1789760793.087048:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544422272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.087077:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544422272:12345-0@lo:400:kworker.0 Request processed in 30us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760793.087083:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1789760793.087101:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422528 00000100:00100000:2.0:1789760793.087104:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422528 00000100:00100000:2.0:1789760793.087130:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422784 00000100:00100000:2.0:1789760793.087133:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422784 00000100:00100000:2.0:1789760793.087153:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789760793.087160:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544422528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.087192:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544422528:12345-0@lo:400:kworker.0 Request processed in 33us (1074us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760793.087198:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789760793.087209:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:2.0:1789760793.087215:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544422784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.087240:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544422784:12345-0@lo:400:kworker.0 Request processed in 26us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760793.087245:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:2.0:1789760793.087276:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544422272:0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.087286:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae806191880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544422784:0@lo:400:kworker.0 00000100:00100000:2.0:1789760793.087294:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544422400:0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.087532:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422144 00000100:00100000:1.0:1789760793.087537:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422144 00000100:00100000:1.0:1789760793.087563:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:1.0:1789760793.087570:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544422144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760793.087615:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544422144:12345-0@lo:400:kworker.0 Request processed in 46us (1403us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.087622:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00080000:1.0:1789760793.087832:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760793.087836:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760793.087839:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760793.089931:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544422144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760793.089949:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae804e34a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544422528:0@lo:400:kworker.0 00002000:00010000:2.0:1789760793.126358:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26660864 -> 26664960 00010000:00010000:2.0:1789760793.126378:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.126397:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.126407:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.126415:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760793.126421:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.126431:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.126449:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.126456:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20465 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.126547:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515835520:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65348us (65559us total) trans 4294973826 rc 0/0 00000100:00100000:2.0:1789760793.126557:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6538 00000100:00100000:2.0:1789760793.126567:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771426777747 time 771526472982 state 0x2 ref 0 00000001:00080000:1.0:1789760793.126937:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973826 is committed 00000100:00100000:2.0:1789760793.130378:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760793.130625:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835648 00000100:00100000:0.0:1789760793.130700:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835648 00000100:00100000:0.0:1789760793.130797:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771426777747 time 771526472982 state 0x0 ref 1 00000100:00100000:0.0:1789760793.130812:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771526733398 time 771526472982 state 0x1 ref 1 00000100:00100000:0.0:1789760793.130898:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771526733398 time 771596185631 state 0x0 ref 1 00000100:00100000:0.0:1789760793.130912:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835648:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760793.130959:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26664960, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:0.0:1789760793.130969:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.130986:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760793.130990:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.130996:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.131010:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.169503:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973827 is committed 00002000:00010000:0.0:1789760793.170046:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26664960 -> 26669056 00010000:00010000:0.0:1789760793.170062:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.170075:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.170084:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.170091:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.170097:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.170105:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.170125:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.170133:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2046c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.170222:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835648:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39309us (39850us total) trans 4294973827 rc 0/0 00000100:00100000:0.0:1789760793.170232:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6539 00000100:00100000:0.0:1789760793.170241:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771526733398 time 771596185631 state 0x2 ref 0 00000100:00100000:1.0:1789760793.174651:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760793.174773:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835776 00000100:00100000:2.0:1789760793.174782:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835776 00000100:00100000:2.0:1789760793.174833:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771526733398 time 771596185631 state 0x0 ref 1 00000100:00100000:2.0:1789760793.174848:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771596446047 time 771596185631 state 0x1 ref 1 00000100:00100000:2.0:1789760793.174906:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771596446047 time 771640194586 state 0x0 ref 1 00000100:00100000:2.0:1789760793.174919:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835776:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760793.174968:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26669056, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:2.0:1789760793.174977:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.174992:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760793.174997:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.175003:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.175017:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760793.222780:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26669056 -> 26673152 00010000:00010000:2.0:1789760793.222799:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.222811:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.222824:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.222832:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760793.222837:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.222846:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.222864:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.222872:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20473 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.222955:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b66d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515835776:12345-192.168.203.46@tcp:4:dd.0 Request processed in 48033us (48308us total) trans 4294973828 rc 0/0 00000100:00100000:2.0:1789760793.222965:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6540 00000100:00100000:2.0:1789760793.222974:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771596446047 time 771640194586 state 0x2 ref 0 00000001:00080000:3.0:1789760793.227988:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973828 is committed 00000100:00100000:1.0:1789760793.228469:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.228625:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515835904 00000100:00100000:3.0:1789760793.228634:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515835904 00000100:00100000:3.0:1789760793.228883:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771596446047 time 771640194586 state 0x0 ref 1 00000100:00100000:3.0:1789760793.228901:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771640455002 time 771640194586 state 0x1 ref 1 00000100:00100000:3.0:1789760793.228934:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771640455002 time 771694223018 state 0x0 ref 1 00000100:00100000:3.0:1789760793.228947:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835904:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760793.228994:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26673152, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:3.0:1789760793.229002:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.229022:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760793.229026:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.229032:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.229046:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.256113:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973829 is committed 00002000:00010000:0.0:1789760793.256461:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26673152 -> 26677248 00010000:00010000:0.0:1789760793.256548:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.256563:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.256573:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.256581:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.256586:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.256608:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.256631:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.256644:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1c81fee/0x69dc3c3d61d2047a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.256934:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515835904:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27984us (28464us total) trans 4294973829 rc 0/0 00000100:00100000:0.0:1789760793.256946:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6541 00000100:00100000:0.0:1789760793.256956:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771640455002 time 771694223018 state 0x2 ref 0 00000100:00100000:1.0:1789760793.260144:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.260591:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836032 00000100:00100000:3.0:1789760793.260610:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836032 00000100:00100000:3.0:1789760793.260665:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771640455002 time 771694223018 state 0x0 ref 1 00000100:00100000:3.0:1789760793.260680:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771694483434 time 771694223018 state 0x1 ref 1 00000100:00100000:3.0:1789760793.260746:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771694483434 time 771726034575 state 0x0 ref 1 00000100:00100000:2.0:1789760793.260999:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515836032:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760793.261049:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26677248, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:2.0:1789760793.261057:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.261074:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760793.261079:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.261085:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.261099:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.292165:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973830 is committed 00002000:00010000:0.0:1789760793.292280:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26677248 -> 26681344 00010000:00010000:0.0:1789760793.292294:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.292308:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.292316:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.292323:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.292327:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.292335:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.292357:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.292363:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20481 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.292449:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515836032:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31528us (32310us total) trans 4294973830 rc 0/0 00000100:00100000:0.0:1789760793.292457:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6542 00000100:00100000:0.0:1789760793.292465:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771694483434 time 771726034575 state 0x2 ref 0 00000100:00100000:1.0:1789760793.299300:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760793.300008:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836160 00000100:00100000:0.0:1789760793.300017:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836160 00000100:00100000:0.0:1789760793.300062:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771694483434 time 771726034575 state 0x0 ref 1 00000100:00100000:0.0:1789760793.300074:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771726294991 time 771726034575 state 0x1 ref 1 00000100:00100000:0.0:1789760793.300096:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771726294991 time 771765387044 state 0x0 ref 1 00000100:00100000:0.0:1789760793.300105:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515836160:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760793.300148:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26681344, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:0.0:1789760793.300155:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.300169:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760793.300172:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.300177:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.300185:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.333293:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973831 is committed 00002000:00010000:0.0:1789760793.333458:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26681344 -> 26685440 00010000:00010000:0.0:1789760793.333489:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.333500:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.333508:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.333523:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.333527:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.333536:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.333555:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.333562:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20488 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.333659:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515836160:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33548us (34361us total) trans 4294973831 rc 0/0 00000100:00100000:0.0:1789760793.333669:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6543 00000100:00100000:0.0:1789760793.333677:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771726294991 time 771765387044 state 0x2 ref 0 00000100:00100000:2.0:1789760793.337285:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.337505:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836288 00000100:00100000:3.0:1789760793.337522:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836288 00000100:00100000:3.0:1789760793.337570:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771726294991 time 771765387044 state 0x0 ref 1 00000100:00100000:3.0:1789760793.337584:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771765647460 time 771765387044 state 0x1 ref 1 00000100:00100000:2.0:1789760793.337791:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771765647460 time 771803077031 state 0x0 ref 1 00000100:00100000:2.0:1789760793.337805:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515836288:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760793.337854:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26685440, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:2.0:1789760793.337863:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.337877:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760793.337881:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.337887:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.337900:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.365549:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26685440 -> 26689536 00010000:00010000:1.0:1789760793.365568:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.365583:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.365586:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973832 is committed 00010000:00010000:1.0:1789760793.365592:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.365611:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.365616:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.365625:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.365644:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.365652:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2048f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.365770:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515836288:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27963us (28488us total) trans 4294973832 rc 0/0 00000100:00100000:1.0:1789760793.365785:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6544 00000100:00100000:1.0:1789760793.365793:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771765647460 time 771803077031 state 0x2 ref 0 00000100:00100000:2.0:1789760793.370896:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760793.371107:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836416 00000100:00100000:1.0:1789760793.371117:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836416 00000100:00100000:1.0:1789760793.371170:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771765647460 time 771803077031 state 0x0 ref 1 00000100:00100000:1.0:1789760793.371185:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771803337447 time 771803077031 state 0x1 ref 1 00000100:00100000:1.0:1789760793.371307:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771803337447 time 771836596026 state 0x0 ref 1 00000100:00100000:1.0:1789760793.371320:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515836416:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760793.371367:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26689536, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:1.0:1789760793.371377:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371392:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760793.371397:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.371404:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371418:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.407967:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973833 is committed 00002000:00010000:1.0:1789760793.408128:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26689536 -> 26693632 00010000:00010000:1.0:1789760793.408141:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.408151:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.408158:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.408162:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.408166:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.408175:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.408199:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.408206:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d20496 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.408605:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515836416:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37270us (37701us total) trans 4294973833 rc 0/0 00000100:00100000:1.0:1789760793.408622:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6545 00000100:00100000:1.0:1789760793.408690:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771803337447 time 771836596026 state 0x2 ref 0 00000100:00100000:3.0:1789760793.417290:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760793.417648:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836544 00000100:00100000:2.0:1789760793.417657:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836544 00000100:00100000:2.0:1789760793.417802:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771803337447 time 771836596026 state 0x0 ref 1 00000100:00100000:2.0:1789760793.417820:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771836856442 time 771836596026 state 0x1 ref 1 00000100:00100000:2.0:1789760793.417849:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771836856442 time 771883138721 state 0x0 ref 1 00000100:00100000:2.0:1789760793.417862:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515836544:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760793.417914:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26693632, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:2.0:1789760793.417923:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.417938:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760793.417943:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.417949:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.417962:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.469000:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973834 is committed 00002000:00010000:2.0:1789760793.469306:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26693632 -> 26697728 00010000:00010000:2.0:1789760793.469323:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.469335:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.469348:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.469355:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760793.469360:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.469369:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.469389:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.469397:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d2049d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.469673:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515836544:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51807us (52388us total) trans 4294973834 rc 0/0 00000100:00100000:2.0:1789760793.469684:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6546 00000100:00100000:2.0:1789760793.469695:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771836856442 time 771883138721 state 0x2 ref 0 00000100:00100000:1.0:1789760793.479476:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.480205:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836672 00000100:00100000:3.0:1789760793.480216:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836672 00000100:00100000:3.0:1789760793.480347:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771836856442 time 771883138721 state 0x0 ref 1 00000100:00100000:3.0:1789760793.480361:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771883399137 time 771883138721 state 0x1 ref 1 00000100:00100000:3.0:1789760793.480391:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771883399137 time 771945680827 state 0x0 ref 1 00000100:00100000:3.0:1789760793.480403:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515836672:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760793.480447:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26697728, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:3.0:1789760793.480457:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.480472:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760793.480477:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.480483:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.480496:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.512461:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26697728 -> 26701824 00010000:00010000:1.0:1789760793.512490:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512504:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512525:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512533:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.512538:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512547:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512563:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512568:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d204a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.512670:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515836672:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32264us (33198us total) trans 4294973835 rc 0/0 00000100:00100000:1.0:1789760793.512682:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6547 00000100:00100000:1.0:1789760793.512691:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771883399137 time 771945680827 state 0x2 ref 0 00000001:00080000:3.0:1789760793.512819:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973835 is committed 00000100:00100000:0.0:1789760793.518124:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760793.518231:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836800 00000100:00100000:2.0:1789760793.518310:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836800 00000100:00100000:2.0:1789760793.518365:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771883399137 time 771945680827 state 0x0 ref 1 00000100:00100000:2.0:1789760793.518380:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771945941243 time 771945680827 state 0x1 ref 1 00000100:00100000:2.0:1789760793.518440:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771945941243 time 771983728488 state 0x0 ref 1 00000100:00100000:2.0:1789760793.518452:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515836800:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760793.518499:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26701824, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:2.0:1789760793.518507:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.518650:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760793.518655:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.518661:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.518675:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.521897:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760793.521945:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760793.522028:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.522160:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837056 00000100:00100000:3.0:1789760793.522170:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837056 00000100:00100000:0.0:1789760793.522231:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515836928 00000100:00100000:0.0:1789760793.522245:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515836928 00000100:00100000:1.0:1789760793.522297:0:5856:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837184 00000100:00100000:1.0:1789760793.522301:0:5856:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837184 00000100:00100000:3.0:1789760793.524622:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 506 00000100:00100000:3.0:1789760793.524636:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515837056:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760793.524729:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2364:x1876699515837056:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 96us (2840us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.524737:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 506 00000100:00100000:1.0:1789760793.526533:0:5856:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 507 00000100:00100000:1.0:1789760793.526546:0:5856:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515837184:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760793.526623:0:5856:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515837184:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 80us (4594us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.526631:0:5856:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 507 00000100:00080000:1.0:1789760793.526652:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760793.526666:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 412 00000100:00100000:1.0:1789760793.526675:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515836928:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760793.526701:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515836928:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 28us (4755us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.526723:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 412 00000001:00080000:3.0:1789760793.560738:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973836 is committed 00002000:00010000:0.0:1789760793.560857:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26701824 -> 26705920 00010000:00010000:0.0:1789760793.560874:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.560888:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.560896:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.560903:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.560907:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.560916:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.560936:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.560944:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d204ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.561034:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806190000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515836800:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42579us (42915us total) trans 4294973836 rc 0/0 00000100:00100000:0.0:1789760793.561044:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6548 00000100:00100000:0.0:1789760793.561820:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771945941243 time 771983728488 state 0x2 ref 0 00000100:00100000:2.0:1789760793.565018:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760793.565134:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837312 00000100:00100000:0.0:1789760793.565144:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837312 00000100:00100000:0.0:1789760793.565195:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771945941243 time 771983728488 state 0x0 ref 1 00000100:00100000:0.0:1789760793.565210:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771983988904 time 771983728488 state 0x1 ref 1 00000100:00100000:0.0:1789760793.565236:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771983988904 time 772030525968 state 0x0 ref 1 00000100:00100000:0.0:1789760793.565250:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837312:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760793.565298:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26705920, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:0.0:1789760793.565307:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.565392:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760793.565399:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.565407:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.565429:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.612044:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973837 is committed 00002000:00010000:1.0:1789760793.612171:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26705920 -> 26710016 00010000:00010000:1.0:1789760793.612187:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.612200:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.612209:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.612221:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.612227:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.612245:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.612264:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.612272:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.612365:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837312:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47113us (47351us total) trans 4294973837 rc 0/0 00000100:00100000:1.0:1789760793.612374:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6549 00000100:00100000:1.0:1789760793.612384:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771983988904 time 772030525968 state 0x2 ref 0 00000100:00100000:1.0:1789760793.617372:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760793.617462:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837440 00000100:00100000:2.0:1789760793.617472:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837440 00000100:00100000:2.0:1789760793.617537:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 771983988904 time 772030525968 state 0x0 ref 1 00000100:00100000:2.0:1789760793.617551:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772030786384 time 772030525968 state 0x1 ref 1 00000100:00100000:2.0:1789760793.617623:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772030786384 time 772082911692 state 0x0 ref 1 00000100:00100000:2.0:1789760793.617637:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837440:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760793.617685:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26710016, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:2.0:1789760793.617695:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.617740:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760793.617745:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.617751:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.617765:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.656309:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973838 is committed 00002000:00010000:3.0:1789760793.656393:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26710016 -> 26714112 00010000:00010000:3.0:1789760793.656407:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.656419:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.656430:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.656437:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760793.656442:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.656452:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.656472:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.656479:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d204b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.656685:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837440:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39038us (39310us total) trans 4294973838 rc 0/0 00000100:00100000:3.0:1789760793.656695:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6550 00000100:00100000:3.0:1789760793.656748:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772030786384 time 772082911692 state 0x2 ref 0 00000100:00100000:0.0:1789760793.663938:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760793.664245:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837568 00000100:00100000:1.0:1789760793.664257:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837568 00000100:00100000:1.0:1789760793.664329:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772030786384 time 772082911692 state 0x0 ref 1 00000100:00100000:1.0:1789760793.664346:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772083172108 time 772082911692 state 0x1 ref 1 00000100:00100000:1.0:1789760793.664379:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772083172108 time 772129668619 state 0x0 ref 1 00000100:00100000:1.0:1789760793.664395:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806191880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837568:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760793.664446:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26714112, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:1.0:1789760793.664455:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.664473:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.664477:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.664485:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.664500:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.688110:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973839 is committed 00002000:00010000:0.0:1789760793.688304:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26714112 -> 26718208 00010000:00010000:0.0:1789760793.688319:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.688332:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.688341:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.688349:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.688354:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.688476:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.688498:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.688506:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d204c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.688692:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806191880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837568:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24295us (24757us total) trans 4294973839 rc 0/0 00000100:00100000:0.0:1789760793.688703:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6551 00000100:00100000:0.0:1789760793.688754:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772083172108 time 772129668619 state 0x2 ref 0 00000100:00100000:2.0:1789760793.696918:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.697814:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837696 00000100:00100000:3.0:1789760793.697825:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837696 00000100:00100000:3.0:1789760793.697880:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772083172108 time 772129668619 state 0x0 ref 1 00000100:00100000:3.0:1789760793.697894:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772129929035 time 772129668619 state 0x1 ref 1 00000100:00100000:3.0:1789760793.697927:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772129929035 time 772163216015 state 0x0 ref 1 00000100:00100000:3.0:1789760793.697940:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837696:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760793.697984:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26718208, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:3.0:1789760793.697993:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.698007:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.698011:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.698018:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.698030:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.732102:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973840 is committed 00002000:00010000:1.0:1789760793.732217:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26718208 -> 26722304 00010000:00010000:1.0:1789760793.732244:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732257:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732266:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732273:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.732279:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732289:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732309:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732318:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d204c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.732415:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837696:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34473us (35500us total) trans 4294973840 rc 0/0 00000100:00100000:1.0:1789760793.732425:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6552 00000100:00100000:1.0:1789760793.732435:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772129929035 time 772163216015 state 0x2 ref 0 00000100:00100000:1.0:1789760793.736768:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.736845:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837824 00000100:00100000:3.0:1789760793.736856:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837824 00000100:00100000:3.0:1789760793.736910:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772129929035 time 772163216015 state 0x0 ref 1 00000100:00100000:3.0:1789760793.736925:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772163476431 time 772163216015 state 0x1 ref 1 00000100:00100000:3.0:1789760793.736947:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772163476431 time 772202237056 state 0x0 ref 1 00000100:00100000:3.0:1789760793.736958:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515837824:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760793.737006:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26722304, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:3.0:1789760793.737015:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.737031:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.737036:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.737043:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.737057:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.764516:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973841 is committed 00002000:00010000:3.0:1789760793.764644:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26722304 -> 26726400 00010000:00010000:3.0:1789760793.764660:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.764678:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.764687:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.764694:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760793.764698:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.764737:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.764758:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.764766:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.764851:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515837824:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27890us (28142us total) trans 4294973841 rc 0/0 00000100:00100000:3.0:1789760793.764862:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6553 00000100:00100000:3.0:1789760793.764871:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772163476431 time 772202237056 state 0x2 ref 0 00000100:00100000:3.0:1789760793.769148:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760793.769532:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515837952 00000100:00100000:0.0:1789760793.769542:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515837952 00000100:00100000:0.0:1789760793.769611:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772163476431 time 772202237056 state 0x0 ref 1 00000100:00100000:0.0:1789760793.769628:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772202497472 time 772202237056 state 0x1 ref 1 00000100:00100000:0.0:1789760793.769663:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772202497472 time 772234952955 state 0x0 ref 1 00000100:00100000:0.0:1789760793.769674:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837952:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760793.769867:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26726400, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:0.0:1789760793.769877:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.769893:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760793.769897:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.769904:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.769919:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.805542:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973842 is committed 00002000:00010000:1.0:1789760793.805701:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26726400 -> 26730496 00010000:00010000:1.0:1789760793.805733:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.805745:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.805755:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.805768:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.805773:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.805783:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.805802:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.805809:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d204d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.805901:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515837952:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36223us (36758us total) trans 4294973842 rc 0/0 00000100:00100000:1.0:1789760793.805910:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6554 00000100:00100000:1.0:1789760793.805920:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772202497472 time 772234952955 state 0x2 ref 0 00000100:00100000:0.0:1789760793.809094:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760793.809261:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838080 00000100:00100000:1.0:1789760793.809272:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838080 00000100:00100000:1.0:1789760793.809324:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772202497472 time 772234952955 state 0x0 ref 1 00000100:00100000:1.0:1789760793.809340:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772235213371 time 772234952955 state 0x1 ref 1 00000100:00100000:1.0:1789760793.809408:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772235213371 time 772274695960 state 0x0 ref 1 00000100:00100000:1.0:1789760793.809425:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515838080:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760793.809474:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26730496, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xd007 00010000:00010000:1.0:1789760793.809482:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.809498:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.809503:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.809509:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.809522:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.866690:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973843 is committed 00002000:00010000:1.0:1789760793.868429:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26730496 -> 26734592 00002000:00010000:1.0:1789760793.868438:0:6712:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:1.0:1789760793.868441:0:6712:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00010000:00010000:1.0:1789760793.868454:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.868468:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.868477:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.868484:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.868490:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.868499:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.868601:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.868611:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.868854:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515838080:12345-192.168.203.46@tcp:4:dd.0 Request processed in 59425us (59763us total) trans 4294973843 rc 0/0 00000100:00100000:1.0:1789760793.868867:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6555 00000100:00100000:1.0:1789760793.868878:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772235213371 time 772274695960 state 0x2 ref 0 00000100:00100000:2.0:1789760793.871868:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760793.871946:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838208 00000100:00100000:0.0:1789760793.871956:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838208 00000100:00100000:0.0:1789760793.872007:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772235213371 time 772274695960 state 0x0 ref 1 00000100:00100000:0.0:1789760793.872023:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772274956376 time 772274695960 state 0x1 ref 1 00000100:00100000:0.0:1789760793.872053:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772274956376 time 772337341903 state 0x0 ref 1 00000100:00100000:0.0:1789760793.872065:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515838208:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760793.872114:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26734592, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:0.0:1789760793.872123:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.872138:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760793.872143:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.872149:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.872162:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.905216:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26734592 -> 26738688 00000001:00080000:1.0:1789760793.905323:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973844 is committed 00010000:00010000:3.0:1789760793.905367:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.905380:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.905389:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.905397:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760793.905402:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.905412:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.905431:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.905438:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d204e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.905541:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80445c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515838208:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33472us (33678us total) trans 4294973844 rc 0/0 00000100:00100000:3.0:1789760793.905555:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6556 00000100:00100000:3.0:1789760793.905565:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772274956376 time 772337341903 state 0x2 ref 0 00000100:00100000:2.0:1789760793.909389:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760793.909645:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838336 00000100:00100000:1.0:1789760793.909655:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838336 00000100:00100000:1.0:1789760793.909813:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772274956376 time 772337341903 state 0x0 ref 1 00000100:00100000:1.0:1789760793.909832:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772337602319 time 772337341903 state 0x1 ref 1 00000100:00100000:1.0:1789760793.909933:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772337602319 time 772375221242 state 0x0 ref 1 00000100:00100000:1.0:1789760793.909947:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838336:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760793.910000:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26738688, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:1.0:1789760793.910009:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.910025:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.910030:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.910036:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.910050:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.924069:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760793.941741:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973845 is committed 00002000:00010000:0.0:1789760793.941855:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26738688 -> 26742784 00010000:00010000:0.0:1789760793.941870:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.941887:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.941895:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.941901:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760793.941905:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.941913:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.941932:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.941939:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd93b89f/0x69dc3c3d61d204ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.942067:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838336:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32119us (32684us total) trans 4294973845 rc 0/0 00000100:00100000:0.0:1789760793.942078:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6557 00000100:00100000:0.0:1789760793.942087:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772337602319 time 772375221242 state 0x2 ref 0 00000100:00100000:2.0:1789760793.945086:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.945160:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838464 00000100:00100000:3.0:1789760793.945170:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838464 00000100:00100000:3.0:1789760793.945219:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772337602319 time 772375221242 state 0x0 ref 1 00000100:00100000:3.0:1789760793.945242:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772375481658 time 772375221242 state 0x1 ref 1 00000100:00100000:3.0:1789760793.945267:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772375481658 time 772410556771 state 0x0 ref 1 00000100:00100000:3.0:1789760793.945279:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838464:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760793.945329:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26742784, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760793.945338:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.945354:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.945358:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.945364:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.945378:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.980687:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26742784 -> 26746880 00010000:00010000:1.0:1789760793.980724:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.980738:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.980749:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.980756:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760793.980761:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.980770:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.980790:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.980799:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d204f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.980894:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515838464:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35612us (35813us total) trans 4294973846 rc 0/0 00000100:00100000:1.0:1789760793.980905:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6558 00000100:00100000:1.0:1789760793.980915:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772375481658 time 772410556771 state 0x2 ref 0 00000001:00080000:2.0:1789760793.980986:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973846 is committed 00000100:00100000:1.0:1789760793.984765:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760793.985174:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838592 00000100:00100000:3.0:1789760793.985184:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838592 00000100:00100000:3.0:1789760793.985295:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772375481658 time 772410556771 state 0x0 ref 1 00000100:00100000:3.0:1789760793.985312:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772410817187 time 772410556771 state 0x1 ref 1 00000100:00100000:3.0:1789760793.985347:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772410817187 time 772450636153 state 0x0 ref 1 00000100:00100000:3.0:1789760793.985360:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838592:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760793.985405:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26746880, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760793.985414:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.985429:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.985434:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.985440:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.985622:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.012915:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973847 is committed 00002000:00010000:3.0:1789760794.013328:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26746880 -> 26750976 00010000:00010000:3.0:1789760794.013344:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.013358:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.013372:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.013379:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.013384:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.013392:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.013412:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.013420:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d204f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.013613:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c3bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838592:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28249us (28853us total) trans 4294973847 rc 0/0 00000100:00100000:3.0:1789760794.013624:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6559 00000100:00100000:3.0:1789760794.013633:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772410817187 time 772450636153 state 0x2 ref 0 00000100:00100000:3.0:1789760794.017115:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760794.017190:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838720 00000100:00100000:1.0:1789760794.017201:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838720 00000100:00100000:1.0:1789760794.017280:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772410817187 time 772450636153 state 0x0 ref 1 00000100:00100000:1.0:1789760794.017296:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772450896569 time 772450636153 state 0x1 ref 1 00000100:00100000:1.0:1789760794.017321:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772450896569 time 772482610868 state 0x0 ref 1 00000100:00100000:1.0:1789760794.017333:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515838720:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760794.017385:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26750976, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:1.0:1789760794.017394:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017415:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760794.017419:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.017426:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017440:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760794.018029:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760794.047072:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26750976 -> 26755072 00010000:00010000:1.0:1789760794.047092:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.047105:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.047113:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.047121:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760794.047125:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.047134:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.047153:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.047161:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d204ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.047379:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515838720:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30044us (30270us total) trans 4294973848 rc 0/0 00000100:00100000:1.0:1789760794.047390:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6560 00000100:00100000:1.0:1789760794.047399:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772450896569 time 772482610868 state 0x2 ref 0 00000001:00080000:1.0:1789760794.047435:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973848 is committed 00000100:00100000:2.0:1789760794.051470:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760794.051610:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838848 00000100:00100000:1.0:1789760794.051619:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838848 00000100:00100000:1.0:1789760794.051673:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772450896569 time 772482610868 state 0x0 ref 1 00000100:00100000:1.0:1789760794.051688:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772482871284 time 772482610868 state 0x1 ref 1 00000100:00100000:1.0:1789760794.051783:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772482871284 time 772517070136 state 0x0 ref 1 00000100:00100000:1.0:1789760794.051795:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838848:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760794.051843:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26755072, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:1.0:1789760794.051852:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.051869:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760794.051873:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.051880:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.051895:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.079731:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973849 is committed 00002000:00010000:1.0:1789760794.079896:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26755072 -> 26759168 00010000:00010000:1.0:1789760794.079912:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.079926:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.079935:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.079943:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760794.079948:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.079957:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.079976:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.079984:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20506 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.080116:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515838848:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28317us (28650us total) trans 4294973849 rc 0/0 00000100:00100000:1.0:1789760794.080127:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6561 00000100:00100000:1.0:1789760794.080136:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772482871284 time 772517070136 state 0x2 ref 0 00000100:00100000:1.0:1789760794.083305:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760794.083427:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515838976 00000100:00100000:0.0:1789760794.083437:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515838976 00000100:00100000:0.0:1789760794.083486:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772482871284 time 772517070136 state 0x0 ref 1 00000100:00100000:0.0:1789760794.083501:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772517330552 time 772517070136 state 0x1 ref 1 00000100:00100000:0.0:1789760794.083538:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772517330552 time 772548826671 state 0x0 ref 1 00000100:00100000:0.0:1789760794.083550:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515838976:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760794.083609:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26759168, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:0.0:1789760794.083618:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.083633:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760794.083638:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.083643:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.083656:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.122652:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973850 is committed 00002000:00010000:0.0:1789760794.122798:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26759168 -> 26763264 00010000:00010000:0.0:1789760794.122813:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.122827:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.122842:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.122850:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.122855:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.122863:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.122883:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.122889:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d2050d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.123015:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515838976:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39462us (39712us total) trans 4294973850 rc 0/0 00000100:00100000:0.0:1789760794.123025:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6562 00000100:00100000:0.0:1789760794.123036:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772517330552 time 772548826671 state 0x2 ref 0 00000100:00100000:1.0:1789760794.126733:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760794.126815:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839104 00000100:00100000:2.0:1789760794.126825:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839104 00000100:00100000:2.0:1789760794.126875:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772517330552 time 772548826671 state 0x0 ref 1 00000100:00100000:2.0:1789760794.126889:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772549087087 time 772548826671 state 0x1 ref 1 00000100:00100000:2.0:1789760794.126922:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772549087087 time 772592211020 state 0x0 ref 1 00000100:00100000:2.0:1789760794.126938:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92bee9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839104:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760794.126983:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26763264, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:2.0:1789760794.126993:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.127008:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760794.127012:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.127019:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.127032:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.152262:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973851 is committed 00002000:00010000:2.0:1789760794.152394:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26763264 -> 26767360 00010000:00010000:2.0:1789760794.152410:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.152424:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.152435:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.152443:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760794.152448:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.152458:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.152478:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.152487:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20514 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.152814:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92bee9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839104:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25873us (26105us total) trans 4294973851 rc 0/0 00000100:00100000:2.0:1789760794.152825:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6563 00000100:00100000:2.0:1789760794.152835:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772549087087 time 772592211020 state 0x2 ref 0 00000100:00100000:1.0:1789760794.157013:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.157114:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839232 00000100:00100000:3.0:1789760794.157123:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839232 00000100:00100000:3.0:1789760794.157173:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772549087087 time 772592211020 state 0x0 ref 1 00000100:00100000:3.0:1789760794.157189:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772592471436 time 772592211020 state 0x1 ref 1 00000100:00100000:3.0:1789760794.157256:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772592471436 time 772622544230 state 0x0 ref 1 00000100:00100000:3.0:1789760794.157268:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839232:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.157315:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26767360, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.157323:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.157338:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760794.157343:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.157349:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.157363:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.185224:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973852 is committed 00002000:00010000:3.0:1789760794.185398:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26767360 -> 26771456 00010000:00010000:3.0:1789760794.185414:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.185427:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.185442:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.185450:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.185454:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.185463:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.185482:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.185490:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d2051b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.185662:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839232:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28390us (28652us total) trans 4294973852 rc 0/0 00000100:00100000:3.0:1789760794.185673:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6564 00000100:00100000:3.0:1789760794.185682:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772592471436 time 772622544230 state 0x2 ref 0 00000100:00100000:0.0:1789760794.189006:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.189089:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839360 00000100:00100000:3.0:1789760794.189099:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839360 00000100:00100000:3.0:1789760794.189148:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772592471436 time 772622544230 state 0x0 ref 1 00000100:00100000:3.0:1789760794.189164:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772622804646 time 772622544230 state 0x1 ref 1 00000100:00100000:3.0:1789760794.189225:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772622804646 time 772654513535 state 0x0 ref 1 00000100:00100000:3.0:1789760794.189247:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839360:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.189293:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26771456, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.189301:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.189316:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760794.189320:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.189326:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.189339:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.225486:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26771456 -> 26775552 00010000:00010000:1.0:1789760794.225506:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.225529:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.225539:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.225546:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760794.225551:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.225561:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.225585:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.225604:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20522 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.225699:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515839360:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36449us (36696us total) trans 4294973853 rc 0/0 00000100:00100000:1.0:1789760794.225745:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6565 00000100:00100000:1.0:1789760794.225756:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772622804646 time 772654513535 state 0x2 ref 0 00000001:00080000:2.0:1789760794.230011:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973853 is committed 00000100:00100000:1.0:1789760794.233897:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760794.234044:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839488 00000100:00100000:2.0:1789760794.234054:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839488 00000100:00100000:2.0:1789760794.234104:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772622804646 time 772654513535 state 0x0 ref 1 00000100:00100000:2.0:1789760794.234119:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772654773951 time 772654513535 state 0x1 ref 1 00000100:00100000:2.0:1789760794.234150:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772654773951 time 772699439682 state 0x0 ref 1 00000100:00100000:2.0:1789760794.234162:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839488:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760794.234208:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26775552, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:2.0:1789760794.234217:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.234300:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760794.234305:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.234311:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.234327:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.265197:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26775552 -> 26779648 00010000:00010000:3.0:1789760794.265218:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265242:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265252:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265260:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.265265:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265275:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265295:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265303:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000089ea0731/0x69dc3c3d61d20529 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.265398:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515839488:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31233us (31505us total) trans 4294973854 rc 0/0 00000100:00100000:3.0:1789760794.265409:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6566 00000100:00100000:3.0:1789760794.265419:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772654773951 time 772699439682 state 0x2 ref 0 00000001:00080000:3.0:1789760794.268682:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973854 is committed 00000100:00100000:0.0:1789760794.268912:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760794.269025:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839616 00000100:00100000:1.0:1789760794.269034:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839616 00000100:00100000:1.0:1789760794.269081:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772654773951 time 772699439682 state 0x0 ref 1 00000100:00100000:1.0:1789760794.269097:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772699700098 time 772699439682 state 0x1 ref 1 00000100:00100000:1.0:1789760794.269129:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772699700098 time 772734416903 state 0x0 ref 1 00000100:00100000:1.0:1789760794.269141:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839616:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760794.269187:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26779648, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:1.0:1789760794.269195:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.269217:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760794.269221:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.269227:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.269251:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.293730:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973855 is committed 00002000:00010000:0.0:1789760794.293885:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26779648 -> 26783744 00010000:00010000:0.0:1789760794.293903:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.293917:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.293927:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.293935:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.293939:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.293948:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.293969:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.293977:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20530 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.294070:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae802f81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515839616:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24925us (25157us total) trans 4294973855 rc 0/0 00000100:00100000:0.0:1789760794.294080:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6567 00000100:00100000:0.0:1789760794.294092:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772699700098 time 772734416903 state 0x2 ref 0 00000100:00100000:0.0:1789760794.297638:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.297771:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839744 00000100:00100000:3.0:1789760794.297782:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839744 00000100:00100000:3.0:1789760794.297837:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772699700098 time 772734416903 state 0x0 ref 1 00000100:00100000:3.0:1789760794.297853:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772734677319 time 772734416903 state 0x1 ref 1 00000100:00100000:3.0:1789760794.297882:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772734677319 time 772763172067 state 0x0 ref 1 00000100:00100000:3.0:1789760794.297893:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515839744:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.297940:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26783744, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.297949:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.297965:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760794.297970:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.297975:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.297989:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.341738:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973856 is committed 00002000:00010000:0.0:1789760794.341873:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26783744 -> 26787840 00010000:00010000:0.0:1789760794.341887:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.341900:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.341908:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.341916:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.341920:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.341928:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.341948:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.341955:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d20537 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.343866:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515839744:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45964us (46225us total) trans 4294973856 rc 0/0 00000100:00100000:0.0:1789760794.343879:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6568 00000100:00100000:0.0:1789760794.343889:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772734677319 time 772763172067 state 0x2 ref 0 00000100:00100000:1.0:1789760794.349853:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760794.349949:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515839872 00000100:00100000:2.0:1789760794.349959:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515839872 00000100:00100000:2.0:1789760794.350011:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772734677319 time 772763172067 state 0x0 ref 1 00000100:00100000:2.0:1789760794.350026:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772763432483 time 772763172067 state 0x1 ref 1 00000100:00100000:2.0:1789760794.350090:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772763432483 time 772815378637 state 0x0 ref 1 00000100:00100000:2.0:1789760794.350104:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515839872:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760794.350154:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26787840, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:2.0:1789760794.350164:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.350179:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760794.350184:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.350190:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.350203:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.389860:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973857 is committed 00002000:00010000:0.0:1789760794.389960:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26787840 -> 26791936 00010000:00010000:0.0:1789760794.389974:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.389986:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.389994:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.390002:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.390007:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.390016:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.390034:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.390042:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2053e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.390132:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c760700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515839872:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40026us (40285us total) trans 4294973857 rc 0/0 00000100:00100000:0.0:1789760794.390142:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6569 00000100:00100000:0.0:1789760794.390153:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772763432483 time 772815378637 state 0x2 ref 0 00000100:00100000:0.0:1789760794.394914:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.395265:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840000 00000100:00100000:3.0:1789760794.395276:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840000 00000100:00100000:3.0:1789760794.395330:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772763432483 time 772815378637 state 0x0 ref 1 00000100:00100000:3.0:1789760794.395345:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772815639053 time 772815378637 state 0x1 ref 1 00000100:00100000:3.0:1789760794.395375:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772815639053 time 772860664420 state 0x0 ref 1 00000100:00100000:3.0:1789760794.395387:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515840000:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.395432:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26791936, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.395442:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.395457:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760794.395463:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.395469:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.395482:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.425830:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973858 is committed 00002000:00010000:3.0:1789760794.426089:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26791936 -> 26796032 00010000:00010000:3.0:1789760794.426105:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.426118:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.426128:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.426136:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.426141:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.426151:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.426171:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.426186:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20545 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.426352:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515840000:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30962us (31442us total) trans 4294973858 rc 0/0 00000100:00100000:3.0:1789760794.426364:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6570 00000100:00100000:3.0:1789760794.426375:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772815639053 time 772860664420 state 0x2 ref 0 00000100:00100000:1.0:1789760794.433381:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.433470:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840128 00000100:00100000:3.0:1789760794.433480:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840128 00000100:00100000:3.0:1789760794.433528:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772815639053 time 772860664420 state 0x0 ref 1 00000100:00100000:3.0:1789760794.433542:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772860924836 time 772860664420 state 0x1 ref 1 00000100:00100000:3.0:1789760794.433571:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772860924836 time 772898860105 state 0x0 ref 1 00000100:00100000:3.0:1789760794.433583:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515840128:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.433643:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26796032, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.433652:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433667:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760794.433671:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.433678:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433695:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.469828:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973859 is committed 00002000:00010000:3.0:1789760794.469942:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26796032 -> 26800128 00010000:00010000:3.0:1789760794.469959:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.469972:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.469981:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.469989:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.469993:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.470003:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.470021:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.470041:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2054c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.470383:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b10a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515840128:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36795us (37004us total) trans 4294973859 rc 0/0 00000100:00100000:3.0:1789760794.470394:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6571 00000100:00100000:3.0:1789760794.470404:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772860924836 time 772898860105 state 0x2 ref 0 00000100:00100000:1.0:1789760794.479141:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.479283:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840256 00000100:00100000:3.0:1789760794.479293:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840256 00000100:00100000:3.0:1789760794.479343:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772860924836 time 772898860105 state 0x0 ref 1 00000100:00100000:3.0:1789760794.479358:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772899120521 time 772898860105 state 0x1 ref 1 00000100:00100000:3.0:1789760794.479389:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772899120521 time 772944678154 state 0x0 ref 1 00000100:00100000:3.0:1789760794.479400:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515840256:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.479448:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26800128, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.479457:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.479472:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760794.479477:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.479483:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.479496:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.519546:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26800128 -> 26804224 00010000:00010000:3.0:1789760794.519568:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.519569:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973860 is committed 00010000:00010000:3.0:1789760794.519580:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.519599:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.519606:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.519612:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.519629:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.519840:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.519850:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20553 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.520191:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515840256:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40787us (41053us total) trans 4294973860 rc 0/0 00000100:00100000:3.0:1789760794.520205:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6572 00000100:00100000:3.0:1789760794.520214:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772899120521 time 772944678154 state 0x2 ref 0 00000100:00100000:0.0:1789760794.526014:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760794.526060:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840384 00000100:00100000:1.0:1789760794.526069:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840384 00000100:00100000:1.0:1789760794.526120:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772899120521 time 772944678154 state 0x0 ref 1 00000100:00100000:1.0:1789760794.526133:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772944938570 time 772944678154 state 0x1 ref 1 00000100:00100000:1.0:1789760794.526163:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772944938570 time 772991452463 state 0x0 ref 1 00000100:00100000:1.0:1789760794.526176:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515840384:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760794.526318:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26804224, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:1.0:1789760794.526328:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.526345:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760794.526349:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.526356:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.526370:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.554096:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973861 is committed 00002000:00010000:2.0:1789760794.554196:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26804224 -> 26808320 00010000:00010000:2.0:1789760794.554211:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.554223:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.554233:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.554250:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760794.554255:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.554263:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.554286:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.554294:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2055a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.554392:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515840384:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28212us (28381us total) trans 4294973861 rc 0/0 00000100:00100000:2.0:1789760794.554402:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6573 00000100:00100000:2.0:1789760794.554411:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772944938570 time 772991452463 state 0x2 ref 0 00000100:00100000:1.0:1789760794.560381:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760794.560514:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840512 00000100:00100000:0.0:1789760794.560524:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840512 00000100:00100000:0.0:1789760794.560580:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772944938570 time 772991452463 state 0x0 ref 1 00000100:00100000:0.0:1789760794.560605:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772991712879 time 772991452463 state 0x1 ref 1 00000100:00100000:0.0:1789760794.560636:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772991712879 time 773025925094 state 0x0 ref 1 00000100:00100000:0.0:1789760794.560648:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515840512:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760794.560692:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26808320, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:0.0:1789760794.560701:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.560736:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.560741:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.560747:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.560762:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.603815:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973862 is committed 00002000:00010000:3.0:1789760794.603976:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26808320 -> 26812416 00010000:00010000:3.0:1789760794.603990:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.604003:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.604013:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.604020:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760794.604036:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.604045:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.604064:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.604072:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d776ff72/0x69dc3c3d61d20561 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.604168:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8057b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515840512:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43518us (43791us total) trans 4294973862 rc 0/0 00000100:00100000:3.0:1789760794.604180:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6574 00000100:00100000:3.0:1789760794.604189:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772991712879 time 773025925094 state 0x2 ref 0 00000100:00100000:0.0:1789760794.611942:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760794.612643:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840640 00000100:00100000:2.0:1789760794.612653:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840640 00000100:00100000:2.0:1789760794.612748:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 772991712879 time 773025925094 state 0x0 ref 1 00000100:00100000:2.0:1789760794.612765:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773026185510 time 773025925094 state 0x1 ref 1 00000100:00100000:2.0:1789760794.612794:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773026185510 time 773078081879 state 0x0 ref 1 00000100:00100000:0.0:1789760794.612875:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515840640:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760794.612921:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26812416, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:0.0:1789760794.612930:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.612946:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.612950:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.612957:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.612970:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.645784:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973863 is committed 00002000:00010000:2.0:1789760794.645946:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26812416 -> 26816512 00010000:00010000:2.0:1789760794.645962:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.645973:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.645981:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.645988:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760794.645992:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.646004:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.646022:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.646030:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ad945d6/0x69dc3c3d61d20568 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.646130:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515840640:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33319us (34191us total) trans 4294973863 rc 0/0 00000100:00100000:2.0:1789760794.646141:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6575 00000100:00100000:2.0:1789760794.646150:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773026185510 time 773078081879 state 0x2 ref 0 00000100:00100000:0.0:1789760794.660322:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.661037:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840768 00000100:00100000:3.0:1789760794.661053:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840768 00000100:00100000:3.0:1789760794.661137:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773026185510 time 773078081879 state 0x0 ref 1 00000100:00100000:3.0:1789760794.661156:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773078342295 time 773078081879 state 0x1 ref 1 00000100:00100000:3.0:1789760794.661201:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773078342295 time 773126489800 state 0x0 ref 1 00000100:00100000:3.0:1789760794.661218:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515840768:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.661295:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26816512, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.661306:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.661328:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.661333:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.661341:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.661359:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.701519:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973864 is committed 00002000:00010000:0.0:1789760794.701931:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26816512 -> 26820608 00010000:00010000:0.0:1789760794.701953:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.701968:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.701978:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.701987:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.701993:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.702003:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.702026:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.702034:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2056f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.702158:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515840768:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40936us (41847us total) trans 4294973864 rc 0/0 00000100:00100000:0.0:1789760794.702171:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6576 00000100:00100000:0.0:1789760794.702183:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773078342295 time 773126489800 state 0x2 ref 0 00000100:00100000:1.0:1789760794.709260:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.709349:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515840896 00000100:00100000:3.0:1789760794.709364:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515840896 00000100:00100000:3.0:1789760794.709434:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773078342295 time 773126489800 state 0x0 ref 1 00000100:00100000:3.0:1789760794.709454:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773126750216 time 773126489800 state 0x1 ref 1 00000100:00100000:3.0:1789760794.709494:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773126750216 time 773174782904 state 0x0 ref 1 00000100:00100000:3.0:1789760794.709509:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515840896:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.709579:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26820608, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.709604:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.709634:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.709640:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.709647:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.709664:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.747219:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973865 is committed 00002000:00010000:0.0:1789760794.747322:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26820608 -> 26824704 00010000:00010000:0.0:1789760794.747343:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747358:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747368:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747380:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.747385:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747395:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747420:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747428:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20576 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.747560:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515840896:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38047us (38307us total) trans 4294973865 rc 0/0 00000100:00100000:0.0:1789760794.747572:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6577 00000100:00100000:0.0:1789760794.747600:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773126750216 time 773174782904 state 0x2 ref 0 00000100:00100000:1.0:1789760794.750889:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.751307:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841024 00000100:00100000:3.0:1789760794.751319:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841024 00000100:00100000:3.0:1789760794.751377:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773126750216 time 773174782904 state 0x0 ref 1 00000100:00100000:3.0:1789760794.751392:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773175043320 time 773174782904 state 0x1 ref 1 00000100:00100000:3.0:1789760794.751419:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773175043320 time 773216708656 state 0x0 ref 1 00000100:00100000:3.0:1789760794.751433:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515841024:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.751497:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26824704, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.751661:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.751684:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.751688:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.751928:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.751948:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.797021:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973866 is committed 00002000:00010000:2.0:1789760794.797272:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26824704 -> 26828800 00010000:00010000:2.0:1789760794.797293:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.797309:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.797320:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.797333:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760794.797339:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.797350:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.797376:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.797385:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2057d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.797516:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515841024:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46070us (46625us total) trans 4294973866 rc 0/0 00000100:00100000:2.0:1789760794.797528:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6578 00000100:00100000:2.0:1789760794.797541:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773175043320 time 773216708656 state 0x2 ref 0 00000100:00100000:2.0:1789760794.800862:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760794.801051:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841152 00000100:00100000:1.0:1789760794.801064:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841152 00000100:00100000:1.0:1789760794.801137:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773175043320 time 773216708656 state 0x0 ref 1 00000100:00100000:1.0:1789760794.801156:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773216969072 time 773216708656 state 0x1 ref 1 00000100:00100000:1.0:1789760794.801185:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773216969072 time 773266473540 state 0x0 ref 1 00000100:00100000:1.0:1789760794.801199:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515841152:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760794.801268:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26828800, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:1.0:1789760794.801280:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.801301:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.801308:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.801316:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.801334:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760794.854188:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26828800 -> 26832896 00000001:00080000:0.0:1789760794.854198:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973867 is committed 00010000:00010000:2.0:1789760794.854216:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.854227:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.854239:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.854244:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760794.854248:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.854255:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.854275:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.854281:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d20584 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.854390:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515841152:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53188us (53538us total) trans 4294973867 rc 0/0 00000100:00100000:2.0:1789760794.854398:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6579 00000100:00100000:2.0:1789760794.854408:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773216969072 time 773266473540 state 0x2 ref 0 00000100:00100000:0.0:1789760794.858405:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760794.858528:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841280 00000100:00100000:3.0:1789760794.858541:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841280 00000100:00100000:3.0:1789760794.858627:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773216969072 time 773266473540 state 0x0 ref 1 00000100:00100000:3.0:1789760794.858647:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773266733956 time 773266473540 state 0x1 ref 1 00000100:00100000:3.0:1789760794.858683:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773266733956 time 773323971521 state 0x0 ref 1 00000100:00100000:3.0:1789760794.858697:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515841280:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760794.858802:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26832896, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xd007 00010000:00010000:3.0:1789760794.858814:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.858834:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.858840:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.858848:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.858866:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760794.904267:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26832896 -> 26836992 00002000:00010000:0.0:1789760794.904280:0:13072:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760794.904283:0:13072:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00010000:00010000:0.0:1789760794.904301:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.904318:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.904328:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.904337:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760794.904343:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.904353:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.904377:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.904386:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000536c0068/0x69dc3c3d61d2058b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.905057:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515841280:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45800us (46103us total) trans 4294973868 rc 0/0 00000100:00100000:0.0:1789760794.905075:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6580 00000100:00100000:0.0:1789760794.905088:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773266733956 time 773323971521 state 0x2 ref 0 00000001:00080000:1.0:1789760794.905456:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973868 is committed 00000100:00100000:1.0:1789760794.912223:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760794.912581:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841408 00000100:00100000:2.0:1789760794.912605:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841408 00000100:00100000:2.0:1789760794.912864:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773266733956 time 773323971521 state 0x0 ref 1 00000100:00100000:2.0:1789760794.912884:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773324231937 time 773323971521 state 0x1 ref 1 00000100:00100000:2.0:1789760794.913101:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773324231937 time 773378388694 state 0x0 ref 1 00000100:00100000:2.0:1789760794.913119:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515841408:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760794.913190:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26836992, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:2.0:1789760794.913202:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.913222:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.913227:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.913235:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.913252:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.942880:0:5843:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760794.948601:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973869 is committed 00002000:00010000:2.0:1789760794.948751:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26836992 -> 26841088 00010000:00010000:2.0:1789760794.948772:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.948786:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.948796:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.948804:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760794.948810:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.948820:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.948843:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.948851:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20592 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.949127:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515841408:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36005us (36914us total) trans 4294973869 rc 0/0 00000100:00100000:2.0:1789760794.949139:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6581 00000100:00100000:2.0:1789760794.949151:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773324231937 time 773378388694 state 0x2 ref 0 00000100:00100000:2.0:1789760794.952028:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760794.952101:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841536 00000100:00100000:0.0:1789760794.952113:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841536 00000100:00100000:0.0:1789760794.952175:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773324231937 time 773378388694 state 0x0 ref 1 00000100:00100000:0.0:1789760794.952189:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773378649110 time 773378388694 state 0x1 ref 1 00000100:00100000:0.0:1789760794.952217:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773378649110 time 773417505830 state 0x0 ref 1 00000100:00100000:0.0:1789760794.952231:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515841536:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760794.952296:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26841088, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760794.952307:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.952327:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.952332:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.952340:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.952356:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.003619:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973870 is committed 00002000:00010000:1.0:1789760795.003772:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26841088 -> 26845184 00010000:00010000:1.0:1789760795.003793:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.003808:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.003818:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.003826:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760795.003832:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.003842:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.003866:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.003876:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d20599 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.004129:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515841536:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51893us (52109us total) trans 4294973870 rc 0/0 00000100:00100000:1.0:1789760795.004142:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6582 00000100:00100000:1.0:1789760795.004154:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773378649110 time 773417505830 state 0x2 ref 0 00000100:00100000:1.0:1789760795.008514:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.008683:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841664 00000100:00100000:0.0:1789760795.008696:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841664 00000100:00100000:0.0:1789760795.008805:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773378649110 time 773417505830 state 0x0 ref 1 00000100:00100000:0.0:1789760795.008823:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773417766246 time 773417505830 state 0x1 ref 1 00000100:00100000:0.0:1789760795.008857:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773417766246 time 773474146101 state 0x0 ref 1 00000100:00100000:0.0:1789760795.008871:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515841664:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.008945:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26845184, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.008956:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.008986:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760795.008992:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.008999:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.009016:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.048468:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973871 is committed 00002000:00010000:0.0:1789760795.048650:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26845184 -> 26849280 00010000:00010000:0.0:1789760795.048672:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.048688:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.048699:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.048734:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.048741:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.048753:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.048778:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.048785:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.048923:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515841664:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40047us (40425us total) trans 4294973871 rc 0/0 00000100:00100000:0.0:1789760795.048937:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6583 00000100:00100000:0.0:1789760795.048949:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773417766246 time 773474146101 state 0x2 ref 0 00000100:00100000:2.0:1789760795.052981:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.053169:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841792 00000100:00100000:0.0:1789760795.053184:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841792 00000100:00100000:0.0:1789760795.053257:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773417766246 time 773474146101 state 0x0 ref 1 00000100:00100000:0.0:1789760795.053279:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773474406517 time 773474146101 state 0x1 ref 1 00000100:00100000:0.0:1789760795.053317:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773474406517 time 773518605352 state 0x0 ref 1 00000100:00100000:0.0:1789760795.053330:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515841792:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.053395:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26849280, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.053406:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.053427:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760795.053432:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.053440:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.053567:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.111391:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973872 is committed 00002000:00010000:1.0:1789760795.111576:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26849280 -> 26853376 00010000:00010000:1.0:1789760795.111608:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111624:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111635:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111643:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760795.111649:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111658:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111689:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111699:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d205a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.111963:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515841792:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58628us (59259us total) trans 4294973872 rc 0/0 00000100:00100000:1.0:1789760795.111976:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6584 00000100:00100000:1.0:1789760795.111989:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773474406517 time 773518605352 state 0x2 ref 0 00000100:00100000:3.0:1789760795.116313:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760795.116617:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515841920 00000100:00100000:2.0:1789760795.116632:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515841920 00000100:00100000:2.0:1789760795.117186:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773474406517 time 773518605352 state 0x0 ref 1 00000100:00100000:2.0:1789760795.117214:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773518865768 time 773518605352 state 0x1 ref 1 00000100:00100000:1.0:1789760795.117343:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773518865768 time 773582629025 state 0x0 ref 1 00000100:00100000:1.0:1789760795.117363:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515841920:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760795.117438:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26853376, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:1.0:1789760795.117449:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.117472:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760795.117477:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.117485:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.117503:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.135675:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae937c58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544422912:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760795.135806:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760795.135904:0:6709:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544422912 00000100:00100000:2.0:1789760795.135914:0:6709:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544422912 00000100:00100000:2.0:1789760795.135990:0:6709:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789760795.136002:0:6709:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544422912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760795.136035:0:6709:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3831496704 avail 3791650816 00000100:00100000:2.0:1789760795.136085:0:6709:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544422912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760795.136093:0:6709:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789760795.136152:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae937c58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544422912:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760795.136487:0:3323:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760795.136508:0:3323:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544423040:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760795.136543:0:3323:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760795.136574:0:10106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423040 00000100:00100000:1.0:1789760795.136578:0:10106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423040 00000100:00100000:1.0:1789760795.136624:0:10106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760795.136632:0:10106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544423040:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760795.136662:0:10106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760795.136700:0:10106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3323:x1876699544423040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (156us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.136877:0:10106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760795.136921:0:3323:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae92be31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3323:1876699544423040:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1789760795.138270:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760795.138278:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760795.156498:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973873 is committed 00002000:00010000:0.0:1789760795.156520:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26853376 -> 26857472 00010000:00010000:0.0:1789760795.156546:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.156563:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.156574:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.156592:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.156599:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.156611:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.156645:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.156654:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086271bbd/0x69dc3c3d61d205ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.156836:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515841920:12345-192.168.203.46@tcp:4:dd.0 Request processed in 39472us (40530us total) trans 4294973873 rc 0/0 00000100:00100000:0.0:1789760795.156851:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6585 00000100:00100000:0.0:1789760795.156865:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773518865768 time 773582629025 state 0x2 ref 0 00000100:00100000:3.0:1789760795.161135:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760795.161610:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842048 00000100:00100000:2.0:1789760795.161624:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842048 00000100:00100000:2.0:1789760795.161933:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773518865768 time 773582629025 state 0x0 ref 1 00000100:00100000:2.0:1789760795.161956:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773582889441 time 773582629025 state 0x1 ref 1 00000100:00100000:2.0:1789760795.162026:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773582889441 time 773627313201 state 0x0 ref 1 00000100:00100000:2.0:1789760795.162043:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842048:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760795.162112:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26857472, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:2.0:1789760795.162124:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.162145:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760795.162150:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.162159:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.162183:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.207806:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973874 is committed 00002000:00010000:1.0:1789760795.208088:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26857472 -> 26861568 00010000:00010000:1.0:1789760795.208109:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.208125:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.208136:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.208143:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760795.208149:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.208160:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.208185:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.208191:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d205b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.208348:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842048:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46303us (47221us total) trans 4294973874 rc 0/0 00000100:00100000:1.0:1789760795.208361:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6586 00000100:00100000:1.0:1789760795.208375:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773582889441 time 773627313201 state 0x2 ref 0 00000100:00100000:2.0:1789760795.211438:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.211572:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842176 00000100:00100000:0.0:1789760795.211593:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842176 00000100:00100000:0.0:1789760795.211660:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773582889441 time 773627313201 state 0x0 ref 1 00000100:00100000:0.0:1789760795.211678:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773627573617 time 773627313201 state 0x1 ref 1 00000100:00100000:0.0:1789760795.211745:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773627573617 time 773677033015 state 0x0 ref 1 00000100:00100000:0.0:1789760795.211759:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842176:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.211828:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26861568, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.211838:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.211858:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760795.211865:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.211872:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.211887:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.245817:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973875 is committed 00002000:00010000:0.0:1789760795.245987:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26861568 -> 26865664 00010000:00010000:0.0:1789760795.246010:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.246026:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.246040:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.246049:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.246054:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.246065:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.246090:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.246105:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.246266:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842176:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34502us (34836us total) trans 4294973875 rc 0/0 00000100:00100000:0.0:1789760795.246279:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6587 00000100:00100000:0.0:1789760795.246292:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773627573617 time 773677033015 state 0x2 ref 0 00000100:00100000:3.0:1789760795.249576:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760795.249757:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842304 00000100:00100000:1.0:1789760795.249770:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842304 00000100:00100000:1.0:1789760795.249939:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773627573617 time 773677033015 state 0x0 ref 1 00000100:00100000:1.0:1789760795.249958:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773677293431 time 773677033015 state 0x1 ref 1 00000100:00100000:1.0:1789760795.250030:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773677293431 time 773715317406 state 0x0 ref 1 00000100:00100000:1.0:1789760795.250046:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842304:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760795.250140:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26865664, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:1.0:1789760795.250153:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.250173:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760795.250178:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.250186:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.250203:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.282778:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26865664 -> 26869760 00010000:00010000:3.0:1789760795.282799:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.282812:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.282821:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.282834:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760795.282839:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.282848:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.282866:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.282875:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5b3bae2/0x69dc3c3d61d205c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.282966:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515842304:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32918us (33398us total) trans 4294973876 rc 0/0 00000100:00100000:3.0:1789760795.282977:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6588 00000100:00100000:3.0:1789760795.282987:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773677293431 time 773715317406 state 0x2 ref 0 00000001:00080000:0.0:1789760795.283392:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973876 is committed 00000100:00100000:1.0:1789760795.289751:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.289837:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842432 00000100:00100000:0.0:1789760795.289847:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842432 00000100:00100000:0.0:1789760795.289903:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773677293431 time 773715317406 state 0x0 ref 1 00000100:00100000:0.0:1789760795.289918:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773715577822 time 773715317406 state 0x1 ref 1 00000100:00100000:0.0:1789760795.289944:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773715577822 time 773755233351 state 0x0 ref 1 00000100:00100000:0.0:1789760795.289957:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842432:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.290004:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26869760, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.290012:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.290027:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760795.290031:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.290037:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.290050:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.323219:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26869760 -> 26873856 00010000:00010000:0.0:1789760795.323241:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.323254:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.323264:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.323271:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.323275:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.323285:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.323305:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.323312:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.323410:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515842432:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33451us (33701us total) trans 4294973877 rc 0/0 00000100:00100000:0.0:1789760795.323421:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6589 00000100:00100000:0.0:1789760795.323429:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773715577822 time 773755233351 state 0x2 ref 0 00000001:00080000:0.0:1789760795.323478:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973877 is committed 00000100:00100000:1.0:1789760795.330988:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760795.334659:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842560 00000100:00100000:2.0:1789760795.334674:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842560 00000100:00100000:2.0:1789760795.336466:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773715577822 time 773755233351 state 0x0 ref 1 00000100:00100000:2.0:1789760795.336484:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773755493767 time 773755233351 state 0x1 ref 1 00000100:00100000:2.0:1789760795.336573:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773755493767 time 773801861467 state 0x0 ref 1 00000100:00100000:2.0:1789760795.336607:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515842560:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760795.336665:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26873856, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:2.0:1789760795.336674:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.336690:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760795.336694:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.336719:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.336734:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.402320:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973878 is committed 00002000:00010000:0.0:1789760795.402416:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26873856 -> 26877952 00010000:00010000:0.0:1789760795.402432:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.402445:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.402454:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.402461:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.402465:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.402472:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.402491:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.402504:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d205d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.402601:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515842560:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66001us (71667us total) trans 4294973878 rc 0/0 00000100:00100000:0.0:1789760795.402612:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6590 00000100:00100000:0.0:1789760795.402621:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773755493767 time 773801861467 state 0x2 ref 0 00000100:00100000:3.0:1789760795.410208:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.410530:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842688 00000100:00100000:0.0:1789760795.410542:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842688 00000100:00100000:0.0:1789760795.410607:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773755493767 time 773801861467 state 0x0 ref 1 00000100:00100000:0.0:1789760795.410623:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773802121883 time 773801861467 state 0x1 ref 1 00000100:00100000:0.0:1789760795.410686:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773802121883 time 773875974957 state 0x0 ref 1 00000100:00100000:0.0:1789760795.410698:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515842688:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.410776:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26877952, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.410785:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.410801:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760795.410806:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.410812:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.410829:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.474253:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973879 is committed 00002000:00010000:0.0:1789760795.474645:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26877952 -> 26882048 00010000:00010000:0.0:1789760795.474663:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.474678:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.474688:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.474696:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.474702:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.474739:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.474760:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.474767:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.474859:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515842688:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64158us (64765us total) trans 4294973879 rc 0/0 00000100:00100000:0.0:1789760795.474870:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6591 00000100:00100000:0.0:1789760795.474880:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773802121883 time 773875974957 state 0x2 ref 0 00000100:00100000:1.0:1789760795.480925:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760795.481020:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842816 00000100:00100000:2.0:1789760795.481040:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842816 00000100:00100000:2.0:1789760795.481104:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773802121883 time 773875974957 state 0x0 ref 1 00000100:00100000:2.0:1789760795.481121:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773876235373 time 773875974957 state 0x1 ref 1 00000100:00100000:2.0:1789760795.481313:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773876235373 time 773946592464 state 0x0 ref 1 00000100:00100000:2.0:1789760795.481327:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515842816:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760795.481374:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26882048, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:2.0:1789760795.481384:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.481399:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760795.481403:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.481408:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.481422:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.536958:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973880 is committed 00002000:00010000:0.0:1789760795.537060:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26882048 -> 26886144 00010000:00010000:0.0:1789760795.537074:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.537087:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.537097:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.537104:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.537109:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.537119:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.537141:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.537154:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d205df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.537258:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515842816:12345-192.168.203.46@tcp:4:dd.0 Request processed in 55929us (56337us total) trans 4294973880 rc 0/0 00000100:00100000:0.0:1789760795.537269:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6592 00000100:00100000:0.0:1789760795.537280:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773876235373 time 773946592464 state 0x2 ref 0 00000100:00100000:1.0:1789760795.542694:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.542876:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515842944 00000100:00100000:0.0:1789760795.542886:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515842944 00000100:00100000:0.0:1789760795.542940:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773876235373 time 773946592464 state 0x0 ref 1 00000100:00100000:0.0:1789760795.542956:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773946852880 time 773946592464 state 0x1 ref 1 00000100:00100000:0.0:1789760795.542990:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773946852880 time 774008279116 state 0x0 ref 1 00000100:00100000:0.0:1789760795.543002:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842944:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.543173:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26886144, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.543183:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.543199:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.543204:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.543210:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.543240:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.576952:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973881 is committed 00002000:00010000:0.0:1789760795.577075:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26886144 -> 26890240 00010000:00010000:0.0:1789760795.577092:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.577105:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.577113:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.577121:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.577126:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.577135:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.577153:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.577161:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.577262:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515842944:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34256us (34573us total) trans 4294973881 rc 0/0 00000100:00100000:0.0:1789760795.577272:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6593 00000100:00100000:0.0:1789760795.577281:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773946852880 time 774008279116 state 0x2 ref 0 00000100:00100000:3.0:1789760795.581058:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760795.581152:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843072 00000100:00100000:1.0:1789760795.581163:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843072 00000100:00100000:1.0:1789760795.581216:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 773946852880 time 774008279116 state 0x0 ref 1 00000100:00100000:1.0:1789760795.581233:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774008539532 time 774008279116 state 0x1 ref 1 00000100:00100000:1.0:1789760795.581303:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774008539532 time 774046591123 state 0x0 ref 1 00000100:00100000:1.0:1789760795.581316:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515843072:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760795.581366:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26890240, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:1.0:1789760795.581375:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.581391:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.581395:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.581401:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.581415:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.650411:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973882 is committed 00002000:00010000:0.0:1789760795.650598:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26890240 -> 26894336 00010000:00010000:0.0:1789760795.650619:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.650631:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.650640:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.650648:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.650654:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.650663:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.650685:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.650698:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d205ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.650822:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804e36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515843072:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69503us (69768us total) trans 4294973882 rc 0/0 00000100:00100000:0.0:1789760795.650833:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6594 00000100:00100000:0.0:1789760795.650842:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774008539532 time 774046591123 state 0x2 ref 0 00000100:00100000:2.0:1789760795.659596:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.659655:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843200 00000100:00100000:0.0:1789760795.659665:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843200 00000100:00100000:0.0:1789760795.659907:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774008539532 time 774046591123 state 0x0 ref 1 00000100:00100000:0.0:1789760795.659924:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774046851539 time 774046591123 state 0x1 ref 1 00000100:00100000:0.0:1789760795.659958:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774046851539 time 774125246855 state 0x0 ref 1 00000100:00100000:0.0:1789760795.659971:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515843200:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.660017:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26894336, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.660025:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.660041:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.660045:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.660052:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.660071:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.724862:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973883 is committed 00002000:00010000:0.0:1789760795.724930:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26894336 -> 26898432 00010000:00010000:0.0:1789760795.724945:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.724958:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.724968:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.724975:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760795.724980:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.724990:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.725010:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.725018:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d205f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.725109:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515843200:12345-192.168.203.46@tcp:4:dd.0 Request processed in 65137us (65529us total) trans 4294973883 rc 0/0 00000100:00100000:0.0:1789760795.725119:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6595 00000100:00100000:0.0:1789760795.725129:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774046851539 time 774125246855 state 0x2 ref 0 00000100:00100000:1.0:1789760795.730372:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760795.730440:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843328 00000100:00100000:2.0:1789760795.730451:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843328 00000100:00100000:2.0:1789760795.730507:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774046851539 time 774125246855 state 0x0 ref 1 00000100:00100000:2.0:1789760795.730523:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774125507271 time 774125246855 state 0x1 ref 1 00000100:00100000:2.0:1789760795.730553:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774125507271 time 774195842202 state 0x0 ref 1 00000100:00100000:2.0:1789760795.730565:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515843328:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760795.730622:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26898432, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:2.0:1789760795.730632:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730657:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.730662:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.730669:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730682:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.760819:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973884 is committed 00002000:00010000:3.0:1789760795.760885:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26898432 -> 26902528 00010000:00010000:3.0:1789760795.760902:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.760922:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.760931:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.760939:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760795.760944:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.760953:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.760973:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.760980:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d205fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.761067:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515843328:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30499us (30699us total) trans 4294973884 rc 0/0 00000100:00100000:3.0:1789760795.761077:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6596 00000100:00100000:3.0:1789760795.761087:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774125507271 time 774195842202 state 0x2 ref 0 00000100:00100000:3.0:1789760795.766373:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.766448:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843456 00000100:00100000:0.0:1789760795.766464:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843456 00000100:00100000:0.0:1789760795.766534:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774125507271 time 774195842202 state 0x0 ref 1 00000100:00100000:0.0:1789760795.766549:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774196102618 time 774195842202 state 0x1 ref 1 00000100:00100000:0.0:1789760795.766644:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774196102618 time 774231931129 state 0x0 ref 1 00000100:00100000:0.0:1789760795.766658:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515843456:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.766741:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26902528, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.766750:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.766766:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.766771:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.766776:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.766790:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.794618:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973885 is committed 00002000:00010000:1.0:1789760795.794784:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26902528 -> 26906624 00010000:00010000:1.0:1789760795.794799:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794812:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794822:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794830:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760795.794834:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794843:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794867:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794876:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d20602 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.794997:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515843456:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28336us (28636us total) trans 4294973885 rc 0/0 00000100:00100000:1.0:1789760795.795007:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6597 00000100:00100000:1.0:1789760795.795016:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774196102618 time 774231931129 state 0x2 ref 0 00000100:00100000:3.0:1789760795.802055:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760795.802323:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843584 00000100:00100000:1.0:1789760795.802334:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843584 00000100:00100000:1.0:1789760795.802389:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774196102618 time 774231931129 state 0x0 ref 1 00000100:00100000:1.0:1789760795.802406:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774232191545 time 774231931129 state 0x1 ref 1 00000100:00100000:1.0:1789760795.802439:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774232191545 time 774267727579 state 0x0 ref 1 00000100:00100000:1.0:1789760795.802452:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515843584:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760795.802504:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26906624, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:1.0:1789760795.802513:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.802530:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.802535:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.802541:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.802556:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.848567:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973886 is committed 00002000:00010000:1.0:1789760795.848677:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26906624 -> 26910720 00010000:00010000:1.0:1789760795.848693:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.848726:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.848737:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.848745:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760795.848750:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.848759:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.848780:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.848793:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20609 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.848887:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515843584:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46432us (46836us total) trans 4294973886 rc 0/0 00000100:00100000:1.0:1789760795.848897:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6598 00000100:00100000:1.0:1789760795.848907:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774232191545 time 774267727579 state 0x2 ref 0 00000100:00100000:2.0:1789760795.852364:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.852510:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843712 00000100:00100000:0.0:1789760795.852518:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843712 00000100:00100000:0.0:1789760795.852563:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774232191545 time 774267727579 state 0x0 ref 1 00000100:00100000:0.0:1789760795.852584:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774267987995 time 774267727579 state 0x1 ref 1 00000100:00100000:0.0:1789760795.852614:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774267987995 time 774317903116 state 0x0 ref 1 00000100:00100000:0.0:1789760795.852953:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515843712:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.852999:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26910720, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xd007 00010000:00010000:0.0:1789760795.853007:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.853022:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.853027:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.853033:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.853047:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.885021:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973887 is committed 00002000:00010000:2.0:1789760795.885108:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26910720 -> 26914816 00002000:00010000:2.0:1789760795.885113:0:13070:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:2.0:1789760795.885116:0:13070:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00010000:00010000:2.0:1789760795.885128:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885142:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885151:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885158:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760795.885163:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885173:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885191:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.885199:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20610 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.885317:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515843712:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32686us (32957us total) trans 4294973887 rc 0/0 00000100:00100000:2.0:1789760795.885327:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6599 00000100:00100000:2.0:1789760795.885336:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774267987995 time 774317903116 state 0x2 ref 0 00000100:00100000:3.0:1789760795.889630:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760795.889822:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843840 00000100:00100000:1.0:1789760795.889832:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843840 00000100:00100000:1.0:1789760795.889883:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774267987995 time 774317903116 state 0x0 ref 1 00000100:00100000:1.0:1789760795.889897:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774318163532 time 774317903116 state 0x1 ref 1 00000100:00100000:1.0:1789760795.889928:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774318163532 time 774355217584 state 0x0 ref 1 00000100:00100000:1.0:1789760795.889940:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515843840:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760795.889986:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26914816, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:1.0:1789760795.889994:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.890010:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.890014:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.890020:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.890035:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.921099:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26914816 -> 26918912 00000001:00080000:2.0:1789760795.921104:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973888 is committed 00010000:00010000:1.0:1789760795.921119:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.921133:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.921142:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.921150:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760795.921155:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.921165:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.921185:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.921193:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d20617 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.921356:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515843840:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31412us (31730us total) trans 4294973888 rc 0/0 00000100:00100000:1.0:1789760795.921367:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6600 00000100:00100000:1.0:1789760795.921376:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774318163532 time 774355217584 state 0x2 ref 0 00000100:00100000:2.0:1789760795.924590:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760795.924839:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515843968 00000100:00100000:0.0:1789760795.924849:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515843968 00000100:00100000:0.0:1789760795.925091:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774318163532 time 774355217584 state 0x0 ref 1 00000100:00100000:0.0:1789760795.925107:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774355478000 time 774355217584 state 0x1 ref 1 00000100:00100000:0.0:1789760795.925142:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774355478000 time 774390430171 state 0x0 ref 1 00000100:00100000:0.0:1789760795.925319:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515843968:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760795.925370:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26918912, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:0.0:1789760795.925379:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.925395:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.925399:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.925406:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.925420:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.959632:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973889 is committed 00002000:00010000:3.0:1789760795.959775:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26918912 -> 26923008 00010000:00010000:3.0:1789760795.959791:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.959803:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.959811:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.959818:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760795.959823:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.959831:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.959848:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.959854:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d2061e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.959950:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515843968:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34793us (35377us total) trans 4294973889 rc 0/0 00000100:00100000:3.0:1789760795.959961:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6601 00000100:00100000:3.0:1789760795.959970:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774355478000 time 774390430171 state 0x2 ref 0 00000100:00100000:2.0:1789760795.967545:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760795.967806:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844096 00000100:00100000:3.0:1789760795.967821:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844096 00000100:00100000:3.0:1789760795.967876:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774355478000 time 774390430171 state 0x0 ref 1 00000100:00100000:3.0:1789760795.967891:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774390690587 time 774390430171 state 0x1 ref 1 00000100:00100000:3.0:1789760795.967926:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774390690587 time 774433215094 state 0x0 ref 1 00000100:00100000:3.0:1789760795.967942:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844096:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760795.967982:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26923008, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:3.0:1789760795.967989:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.968001:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.968005:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.968010:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.968012:0:7647:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789760795.968026:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.010251:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26923008 -> 26927104 00010000:00010000:3.0:1789760796.010270:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.010284:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.010292:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.010299:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.010304:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.010314:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.010332:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.010339:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20625 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.010425:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515844096:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42486us (42890us total) trans 4294973890 rc 0/0 00000100:00100000:3.0:1789760796.010435:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6602 00000100:00100000:3.0:1789760796.010444:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774390690587 time 774433215094 state 0x2 ref 0 00000001:00080000:2.0:1789760796.010541:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973890 is committed 00000100:00100000:2.0:1789760796.022408:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760796.022741:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844224 00000100:00100000:1.0:1789760796.022752:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844224 00000100:00100000:1.0:1789760796.022860:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774390690587 time 774433215094 state 0x0 ref 1 00000100:00100000:1.0:1789760796.022875:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774433475510 time 774433215094 state 0x1 ref 1 00000100:00100000:1.0:1789760796.022907:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774433475510 time 774488196279 state 0x0 ref 1 00000100:00100000:1.0:1789760796.022919:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844224:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760796.022967:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26927104, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:1.0:1789760796.022976:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.022991:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760796.022996:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.023003:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.023017:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760796.030355:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760796.058976:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973891 is committed 00002000:00010000:2.0:1789760796.059180:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26927104 -> 26931200 00010000:00010000:2.0:1789760796.059195:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.059208:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.059217:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.059225:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.059230:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.059306:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.059329:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.059337:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2062c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.059464:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844224:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36542us (37062us total) trans 4294973891 rc 0/0 00000100:00100000:2.0:1789760796.059476:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6603 00000100:00100000:2.0:1789760796.059487:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774433475510 time 774488196279 state 0x2 ref 0 00000100:00100000:2.0:1789760796.074209:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760796.074375:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844352 00000100:00100000:1.0:1789760796.074385:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844352 00000100:00100000:1.0:1789760796.074441:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774433475510 time 774488196279 state 0x0 ref 1 00000100:00100000:1.0:1789760796.074457:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774488456695 time 774488196279 state 0x1 ref 1 00000100:00100000:3.0:1789760796.074586:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774488456695 time 774539864395 state 0x0 ref 1 00000100:00100000:3.0:1789760796.074601:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515844352:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760796.074649:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26931200, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:3.0:1789760796.074658:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.074674:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760796.074678:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.074685:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.074699:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.118324:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26931200 -> 26935296 00010000:00010000:3.0:1789760796.118346:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.118369:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.118378:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.118386:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.118390:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.118400:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.118419:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.118426:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d20633 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.118525:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515844352:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43922us (44320us total) trans 4294973892 rc 0/0 00000100:00100000:3.0:1789760796.118535:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6604 00000100:00100000:3.0:1789760796.118544:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774488456695 time 774539864395 state 0x2 ref 0 00000001:00080000:2.0:1789760796.119505:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973892 is committed 00000100:00100000:1.0:1789760796.124796:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.125207:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844480 00000100:00100000:2.0:1789760796.125218:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844480 00000100:00100000:2.0:1789760796.125273:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774488456695 time 774539864395 state 0x0 ref 1 00000100:00100000:2.0:1789760796.125288:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774540124811 time 774539864395 state 0x1 ref 1 00000100:00100000:2.0:1789760796.125361:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774540124811 time 774590649696 state 0x0 ref 1 00000100:00100000:2.0:1789760796.125374:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844480:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.125425:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26935296, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.125434:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.125450:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760796.125455:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.125460:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.125479:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.170383:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26935296 -> 26939392 00010000:00010000:3.0:1789760796.170404:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.170416:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.170426:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.170433:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.170439:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.170448:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.170468:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.170475:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2063a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.170677:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515844480:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45300us (45884us total) trans 4294973893 rc 0/0 00000100:00100000:3.0:1789760796.170688:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6605 00000100:00100000:3.0:1789760796.170696:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774540124811 time 774590649696 state 0x2 ref 0 00000001:00080000:2.0:1789760796.171496:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973893 is committed 00000100:00100000:1.0:1789760796.173293:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.173749:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844608 00000100:00100000:2.0:1789760796.173759:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844608 00000100:00100000:2.0:1789760796.173809:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774540124811 time 774590649696 state 0x0 ref 1 00000100:00100000:2.0:1789760796.173824:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774590910112 time 774590649696 state 0x1 ref 1 00000100:00100000:2.0:1789760796.173857:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774590910112 time 774639145230 state 0x0 ref 1 00000100:00100000:2.0:1789760796.173869:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515844608:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.173915:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26939392, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.173924:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.173940:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760796.173945:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.173951:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.173966:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.217685:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973894 is committed 00002000:00010000:2.0:1789760796.217929:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26939392 -> 26943488 00010000:00010000:2.0:1789760796.217945:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.217957:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.217965:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.217972:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.217977:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.217985:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.218003:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.218009:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e2e46f3/0x69dc3c3d61d20641 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.218136:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92be32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515844608:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44265us (44846us total) trans 4294973894 rc 0/0 00000100:00100000:2.0:1789760796.218161:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6606 00000100:00100000:2.0:1789760796.218171:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774590910112 time 774639145230 state 0x2 ref 0 00000100:00100000:2.0:1789760796.221845:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760796.221982:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844736 00000100:00100000:1.0:1789760796.221993:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844736 00000100:00100000:1.0:1789760796.222046:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774590910112 time 774639145230 state 0x0 ref 1 00000100:00100000:1.0:1789760796.222061:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774639405646 time 774639145230 state 0x1 ref 1 00000100:00100000:1.0:1789760796.222090:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774639405646 time 774687379044 state 0x0 ref 1 00000100:00100000:1.0:1789760796.222102:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844736:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760796.222148:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26943488, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:1.0:1789760796.222157:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.222172:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760796.222177:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.222183:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.222202:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.250126:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973895 is committed 00002000:00010000:2.0:1789760796.250225:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26943488 -> 26947584 00010000:00010000:2.0:1789760796.250242:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.250256:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.250265:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.250272:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.250277:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.250286:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.250305:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.250313:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d20648 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.250653:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844736:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28548us (28813us total) trans 4294973895 rc 0/0 00000100:00100000:2.0:1789760796.250664:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6607 00000100:00100000:2.0:1789760796.250674:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774639405646 time 774687379044 state 0x2 ref 0 00000100:00100000:3.0:1789760796.253461:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760796.253610:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844864 00000100:00100000:0.0:1789760796.253621:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844864 00000100:00100000:0.0:1789760796.253800:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774639405646 time 774687379044 state 0x0 ref 1 00000100:00100000:0.0:1789760796.253815:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774687639460 time 774687379044 state 0x1 ref 1 00000100:00100000:0.0:1789760796.253855:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774687639460 time 774719135565 state 0x0 ref 1 00000100:00100000:0.0:1789760796.253869:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515844864:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760796.253917:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26947584, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:0.0:1789760796.253925:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.253940:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760796.253944:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.253950:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.253974:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.279477:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26947584 -> 26951680 00000001:00080000:2.0:1789760796.279491:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973896 is committed 00010000:00010000:0.0:1789760796.279507:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.279520:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.279531:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.279539:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bdc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760796.279544:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.279553:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.279584:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.279593:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acac946f/0x69dc3c3d61d2064f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.279674:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515844864:12345-192.168.203.46@tcp:4:dd.0 Request processed in 25804us (26218us total) trans 4294973896 rc 0/0 00000100:00100000:0.0:1789760796.279685:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6608 00000100:00100000:0.0:1789760796.279696:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774687639460 time 774719135565 state 0x2 ref 0 00000100:00100000:1.0:1789760796.284495:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.284646:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515844992 00000100:00100000:2.0:1789760796.284657:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515844992 00000100:00100000:2.0:1789760796.284837:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774687639460 time 774719135565 state 0x0 ref 1 00000100:00100000:2.0:1789760796.284853:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774719395981 time 774719135565 state 0x1 ref 1 00000100:00100000:2.0:1789760796.284914:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774719395981 time 774750203166 state 0x0 ref 1 00000100:00100000:2.0:1789760796.284927:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae806249c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844992:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.284977:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26951680, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.284986:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.285002:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760796.285007:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.285013:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.285026:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.317314:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973897 is committed 00002000:00010000:2.0:1789760796.317398:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26951680 -> 26955776 00010000:00010000:2.0:1789760796.317413:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.317426:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.317435:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.317442:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.317447:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.317456:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.317475:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.317482:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20656 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.321106:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae806249c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515844992:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36167us (36609us total) trans 4294973897 rc 0/0 00000100:00100000:2.0:1789760796.321126:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6609 00000100:00100000:2.0:1789760796.321138:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774719395981 time 774750203166 state 0x2 ref 0 00000100:00100000:2.0:1789760796.325082:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.325262:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845120 00000100:00100000:3.0:1789760796.325274:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845120 00000100:00100000:3.0:1789760796.325332:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774719395981 time 774750203166 state 0x0 ref 1 00000100:00100000:3.0:1789760796.325347:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774750463582 time 774750203166 state 0x1 ref 1 00000100:00100000:3.0:1789760796.325428:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774750463582 time 774790716744 state 0x0 ref 1 00000100:00100000:3.0:1789760796.325443:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845120:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760796.325501:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26955776, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:3.0:1789760796.325511:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.325527:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760796.325531:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.325537:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.325550:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.394640:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973898 is committed 00002000:00010000:2.0:1789760796.394766:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26955776 -> 26959872 00010000:00010000:2.0:1789760796.394782:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.394794:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.394802:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.394808:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.394813:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.394822:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.394841:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.394849:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063898613/0x69dc3c3d61d2065d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.394947:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845120:12345-192.168.203.46@tcp:4:dd.0 Request processed in 69501us (69871us total) trans 4294973898 rc 0/0 00000100:00100000:2.0:1789760796.394956:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6610 00000100:00100000:2.0:1789760796.394966:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774750463582 time 774790716744 state 0x2 ref 0 00000100:00100000:1.0:1789760796.399153:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.399287:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845248 00000100:00100000:3.0:1789760796.399297:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845248 00000100:00100000:3.0:1789760796.399351:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774750463582 time 774790716744 state 0x0 ref 1 00000100:00100000:3.0:1789760796.399366:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774790977160 time 774790716744 state 0x1 ref 1 00000100:00100000:3.0:1789760796.399397:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774790977160 time 774864686286 state 0x0 ref 1 00000100:00100000:3.0:1789760796.399410:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845248:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760796.399466:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26959872, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:3.0:1789760796.399483:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.399505:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760796.399510:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.399517:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.399529:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.445471:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973899 is committed 00002000:00010000:3.0:1789760796.445778:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26959872 -> 26963968 00010000:00010000:3.0:1789760796.445795:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.445818:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.445827:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.445833:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.445837:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.445846:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.445864:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.445871:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20664 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.445961:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80624b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845248:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46549us (46813us total) trans 4294973899 rc 0/0 00000100:00100000:3.0:1789760796.445971:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6611 00000100:00100000:3.0:1789760796.445981:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774790977160 time 774864686286 state 0x2 ref 0 00000100:00100000:0.0:1789760796.453413:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.453580:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845376 00000100:00100000:2.0:1789760796.453590:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845376 00000100:00100000:2.0:1789760796.453644:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774790977160 time 774864686286 state 0x0 ref 1 00000100:00100000:2.0:1789760796.453659:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774864946702 time 774864686286 state 0x1 ref 1 00000100:00100000:2.0:1789760796.453977:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774864946702 time 774919264386 state 0x0 ref 1 00000100:00100000:2.0:1789760796.453990:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845376:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.454039:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26963968, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.454049:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.454065:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760796.454069:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.454076:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.454092:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.481319:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973900 is committed 00002000:00010000:2.0:1789760796.481405:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26963968 -> 26968064 00010000:00010000:2.0:1789760796.481421:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481434:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481444:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481451:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.481456:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481566:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481662:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481672:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d2066b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.481815:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845376:12345-192.168.203.46@tcp:4:dd.0 Request processed in 27822us (28405us total) trans 4294973900 rc 0/0 00000100:00100000:2.0:1789760796.481832:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6612 00000100:00100000:2.0:1789760796.481853:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774864946702 time 774919264386 state 0x2 ref 0 00000100:00100000:0.0:1789760796.489699:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.490059:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845504 00000100:00100000:2.0:1789760796.490070:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845504 00000100:00100000:2.0:1789760796.490124:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774864946702 time 774919264386 state 0x0 ref 1 00000100:00100000:2.0:1789760796.490150:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774919524802 time 774919264386 state 0x1 ref 1 00000100:00100000:2.0:1789760796.490181:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774919524802 time 774955470581 state 0x0 ref 1 00000100:00100000:2.0:1789760796.490193:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515845504:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.490252:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26968064, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.490262:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.490278:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760796.490283:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.490289:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.490303:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.524430:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26968064 -> 26972160 00010000:00010000:3.0:1789760796.524450:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.524462:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.524472:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.524478:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.524483:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.524501:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.524525:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.524533:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000728de842/0x69dc3c3d61d20672 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.524641:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2363:x1876699515845504:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34438us (34948us total) trans 4294973901 rc 0/0 00000100:00100000:3.0:1789760796.524651:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6613 00000100:00100000:3.0:1789760796.524660:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774919524802 time 774955470581 state 0x2 ref 0 00000001:00080000:2.0:1789760796.524789:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973901 is committed 00000100:00100000:1.0:1789760796.541100:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760796.541175:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845632 00000100:00100000:0.0:1789760796.541183:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845632 00000100:00100000:0.0:1789760796.541232:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774919524802 time 774955470581 state 0x0 ref 1 00000100:00100000:0.0:1789760796.541247:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774955730997 time 774955470581 state 0x1 ref 1 00000100:00100000:0.0:1789760796.541282:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774955730997 time 775006571199 state 0x0 ref 1 00000100:00100000:0.0:1789760796.541293:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515845632:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760796.541341:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26972160, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:0.0:1789760796.541350:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.541366:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.541370:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.541376:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.541392:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.599484:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973902 is committed 00002000:00010000:3.0:1789760796.599871:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26972160 -> 26976256 00010000:00010000:3.0:1789760796.599886:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.599899:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.599907:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.599914:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.599919:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.599932:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.599948:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.599956:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000061aaa5b/0x69dc3c3d61d20679 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.600147:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515845632:12345-192.168.203.46@tcp:4:dd.0 Request processed in 58850us (59051us total) trans 4294973902 rc 0/0 00000100:00100000:3.0:1789760796.600159:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6614 00000100:00100000:3.0:1789760796.600169:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774955730997 time 775006571199 state 0x2 ref 0 00000100:00100000:1.0:1789760796.606967:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.607072:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845760 00000100:00100000:2.0:1789760796.607081:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845760 00000100:00100000:2.0:1789760796.607134:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 774955730997 time 775006571199 state 0x0 ref 1 00000100:00100000:2.0:1789760796.607159:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775006831615 time 775006571199 state 0x1 ref 1 00000100:00100000:2.0:1789760796.607191:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775006831615 time 775072480280 state 0x0 ref 1 00000100:00100000:2.0:1789760796.607202:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515845760:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.607247:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26976256, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.607256:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.607271:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.607276:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.607282:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.607295:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.631953:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26976256 -> 26980352 00010000:00010000:3.0:1789760796.631972:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.631985:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.631994:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.632002:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.632008:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.632017:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.632035:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.632042:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004420f2f0/0x69dc3c3d61d20680 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.632136:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8029c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515845760:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24931us (25175us total) trans 4294973903 rc 0/0 00000100:00100000:3.0:1789760796.632147:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6615 00000100:00100000:3.0:1789760796.632156:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775006831615 time 775072480280 state 0x2 ref 0 00000001:00080000:2.0:1789760796.632360:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973903 is committed 00000100:00100000:0.0:1789760796.642451:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.642820:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515845888 00000100:00100000:3.0:1789760796.642830:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515845888 00000100:00100000:3.0:1789760796.642881:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775006831615 time 775072480280 state 0x0 ref 1 00000100:00100000:3.0:1789760796.642897:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775072740696 time 775072480280 state 0x1 ref 1 00000100:00100000:3.0:1789760796.642935:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775072740696 time 775108224694 state 0x0 ref 1 00000100:00100000:3.0:1789760796.642946:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515845888:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760796.642991:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26980352, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:3.0:1789760796.643000:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.643015:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760796.643019:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.643025:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.643038:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.669060:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973904 is committed 00002000:00010000:2.0:1789760796.669179:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26980352 -> 26984448 00010000:00010000:2.0:1789760796.669196:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.669210:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.669221:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.669228:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.669235:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.669245:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.669269:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.669278:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dacb491/0x69dc3c3d61d20687 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.669389:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515845888:12345-192.168.203.46@tcp:4:dd.0 Request processed in 26438us (26942us total) trans 4294973904 rc 0/0 00000100:00100000:2.0:1789760796.669401:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6616 00000100:00100000:2.0:1789760796.669411:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775072740696 time 775108224694 state 0x2 ref 0 00000100:00100000:0.0:1789760796.677517:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.677655:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846016 00000100:00100000:3.0:1789760796.677665:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846016 00000100:00100000:3.0:1789760796.677791:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775072740696 time 775108224694 state 0x0 ref 1 00000100:00100000:3.0:1789760796.677807:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775108485110 time 775108224694 state 0x1 ref 1 00000100:00100000:2.0:1789760796.677968:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775108485110 time 775143254939 state 0x0 ref 1 00000100:00100000:2.0:1789760796.677984:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515846016:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.678038:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26984448, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.678047:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.678062:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.678066:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.678073:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.678086:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.708184:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973905 is committed 00002000:00010000:3.0:1789760796.708428:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26984448 -> 26988544 00010000:00010000:3.0:1789760796.708444:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.708457:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.708467:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.708475:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760796.708481:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.708589:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.708610:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.708618:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be3eaaeb/0x69dc3c3d61d2068e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.708746:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515846016:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30726us (31198us total) trans 4294973905 rc 0/0 00000100:00100000:3.0:1789760796.708757:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6617 00000100:00100000:3.0:1789760796.708767:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775108485110 time 775143254939 state 0x2 ref 0 00000100:00100000:0.0:1789760796.716805:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.716937:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846144 00000100:00100000:2.0:1789760796.716949:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846144 00000100:00100000:2.0:1789760796.717009:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775108485110 time 775143254939 state 0x0 ref 1 00000100:00100000:2.0:1789760796.717025:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775143515355 time 775143254939 state 0x1 ref 1 00000100:00100000:2.0:1789760796.717092:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775143515355 time 775182380021 state 0x0 ref 1 00000100:00100000:2.0:1789760796.717104:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515846144:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.717154:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26988544, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.717163:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.717179:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.717186:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.717192:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.717206:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.757884:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973906 is committed 00002000:00010000:1.0:1789760796.758136:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26988544 -> 26992640 00010000:00010000:1.0:1789760796.758152:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.758165:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.758174:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.758181:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760796.758186:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.758196:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.758294:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.758303:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067ff698e/0x69dc3c3d61d20695 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.758396:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515846144:12345-192.168.203.46@tcp:4:dd.0 Request processed in 41287us (41594us total) trans 4294973906 rc 0/0 00000100:00100000:1.0:1789760796.758406:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6618 00000100:00100000:1.0:1789760796.758415:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775143515355 time 775182380021 state 0x2 ref 0 00000100:00100000:0.0:1789760796.762257:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760796.762400:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846272 00000100:00100000:2.0:1789760796.762411:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846272 00000100:00100000:2.0:1789760796.762467:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775143515355 time 775182380021 state 0x0 ref 1 00000100:00100000:2.0:1789760796.762485:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775182640437 time 775182380021 state 0x1 ref 1 00000100:00100000:2.0:1789760796.762527:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775182640437 time 775227816610 state 0x0 ref 1 00000100:00100000:2.0:1789760796.762539:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515846272:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.762600:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26992640, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.762609:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.762624:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.762629:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.762635:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.762648:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.800580:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973907 is committed 00002000:00010000:2.0:1789760796.801212:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26992640 -> 26996736 00010000:00010000:2.0:1789760796.801228:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.801305:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.801314:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.801321:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.801326:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.801335:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.801354:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.801362:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d2069c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.801479:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515846272:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38938us (39227us total) trans 4294973907 rc 0/0 00000100:00100000:2.0:1789760796.801496:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6619 00000100:00100000:2.0:1789760796.801505:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775182640437 time 775227816610 state 0x2 ref 0 00000100:00100000:0.0:1789760796.812423:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.814466:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846400 00000100:00100000:3.0:1789760796.814480:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846400 00000100:00100000:2.0:1789760796.814657:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775182640437 time 775227816610 state 0x0 ref 1 00000100:00100000:2.0:1789760796.814674:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775228077026 time 775227816610 state 0x1 ref 1 00000100:00100000:2.0:1789760796.814765:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775228077026 time 775280053361 state 0x0 ref 1 00000100:00100000:2.0:1789760796.814779:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515846400:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.814829:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 26996736, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xd007 00010000:00010000:2.0:1789760796.814838:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.814854:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.814858:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.814864:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.814877:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760796.850766:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 26996736 -> 27000832 00002000:00010000:2.0:1789760796.850774:0:6711:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:2.0:1789760796.850777:0:6711:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00010000:00010000:2.0:1789760796.850792:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.850805:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.850813:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.850819:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.850824:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.850834:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.850854:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.850861:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.850958:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515846400:12345-192.168.203.46@tcp:4:dd.0 Request processed in 36178us (38540us total) trans 4294973908 rc 0/0 00000100:00100000:2.0:1789760796.850968:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6620 00000100:00100000:2.0:1789760796.850978:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775228077026 time 775280053361 state 0x2 ref 0 00000001:00080000:0.0:1789760796.851056:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973908 is committed 00000100:00100000:0.0:1789760796.855389:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.855660:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846528 00000100:00100000:3.0:1789760796.855669:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846528 00000100:00100000:3.0:1789760796.855849:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775228077026 time 775280053361 state 0x0 ref 1 00000100:00100000:3.0:1789760796.855864:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775280313777 time 775280053361 state 0x1 ref 1 00000100:00100000:2.0:1789760796.855960:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775280313777 time 775321247233 state 0x0 ref 1 00000100:00100000:2.0:1789760796.855975:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515846528:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.856020:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27000832, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760796.856029:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856045:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.856049:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.856056:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.856070:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.887930:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973909 is committed 00002000:00010000:2.0:1789760796.888063:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27000832 -> 27004928 00010000:00010000:2.0:1789760796.888078:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888092:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888100:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888108:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.888113:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888122:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888143:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888151:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.888241:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515846528:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32264us (32856us total) trans 4294973909 rc 0/0 00000100:00100000:2.0:1789760796.888251:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6621 00000100:00100000:2.0:1789760796.888261:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775280313777 time 775321247233 state 0x2 ref 0 00000100:00100000:1.0:1789760796.891438:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760796.891550:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846656 00000100:00100000:0.0:1789760796.891562:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846656 00000100:00100000:0.0:1789760796.891623:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775280313777 time 775321247233 state 0x0 ref 1 00000100:00100000:0.0:1789760796.891639:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775321507649 time 775321247233 state 0x1 ref 1 00000100:00100000:0.0:1789760796.891670:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775321507649 time 775356959935 state 0x0 ref 1 00000100:00100000:0.0:1789760796.891683:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515846656:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760796.891972:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27004928, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:0.0:1789760796.891981:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.891998:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.892002:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.892007:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.892021:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.933718:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973910 is committed 00002000:00010000:0.0:1789760796.933747:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27004928 -> 27009024 00010000:00010000:0.0:1789760796.933764:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.933778:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.933787:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.933794:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760796.933799:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.933808:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.933829:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.933837:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d206b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.933927:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93bfd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515846656:12345-192.168.203.46@tcp:4:dd.0 Request processed in 42242us (42496us total) trans 4294973910 rc 0/0 00000100:00100000:0.0:1789760796.933936:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6622 00000100:00100000:0.0:1789760796.933945:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775321507649 time 775356959935 state 0x2 ref 0 00000100:00100000:0.0:1789760796.941746:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760796.941799:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846784 00000100:00100000:3.0:1789760796.941810:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846784 00000100:00100000:3.0:1789760796.941865:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775321507649 time 775356959935 state 0x0 ref 1 00000100:00100000:3.0:1789760796.941880:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775357220351 time 775356959935 state 0x1 ref 1 00000100:00100000:2.0:1789760796.942029:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775357220351 time 775407316044 state 0x0 ref 1 00000100:00100000:2.0:1789760796.942045:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae8010e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515846784:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760796.942108:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27009024, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760796.942116:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.942133:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.942137:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.942144:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.942157:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.987969:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973911 is committed 00002000:00010000:2.0:1789760796.988070:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27009024 -> 27013120 00010000:00010000:2.0:1789760796.988085:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.988097:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.988106:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.988113:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760796.988118:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.988127:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.988145:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.988153:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.988242:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae8010e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515846784:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46195us (46535us total) trans 4294973911 rc 0/0 00000100:00100000:2.0:1789760796.988252:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6623 00000100:00100000:2.0:1789760796.988262:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775357220351 time 775407316044 state 0x2 ref 0 00010000:00010000:0.0:1789760796.989903:0:7648:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760797.002040:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.005874:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515846912 00000100:00100000:2.0:1789760797.005889:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515846912 00000100:00100000:3.0:1789760797.006140:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775357220351 time 775407316044 state 0x0 ref 1 00000100:00100000:3.0:1789760797.006157:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775407576460 time 775407316044 state 0x1 ref 1 00000100:00100000:3.0:1789760797.006191:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775407576460 time 775471479661 state 0x0 ref 1 00000100:00100000:3.0:1789760797.006205:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515846912:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.006255:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27013120, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.006265:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.006281:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760797.006286:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.006291:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.006305:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.041560:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973912 is committed 00002000:00010000:1.0:1789760797.041815:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27013120 -> 27017216 00010000:00010000:1.0:1789760797.041828:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.041838:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.041918:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.041925:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760797.041930:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.041939:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.041958:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.041966:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044426ff5/0x69dc3c3d61d206bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.042060:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515846912:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35854us (40024us total) trans 4294973912 rc 0/0 00000100:00100000:1.0:1789760797.042070:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6624 00000100:00100000:1.0:1789760797.042090:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775407576460 time 775471479661 state 0x2 ref 0 00000100:00100000:0.0:1789760797.048398:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.052291:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847040 00000100:00100000:2.0:1789760797.052306:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847040 00000100:00100000:2.0:1789760797.052370:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775407576460 time 775471479661 state 0x0 ref 1 00000100:00100000:2.0:1789760797.052385:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775471740077 time 775471479661 state 0x1 ref 1 00000100:00100000:2.0:1789760797.052419:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775471740077 time 775517708412 state 0x0 ref 1 00000100:00100000:2.0:1789760797.052433:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515847040:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.052481:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27017216, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760797.052499:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.052514:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760797.052519:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.052526:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.052540:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.086195:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973913 is committed 00002000:00010000:3.0:1789760797.086374:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27017216 -> 27021312 00010000:00010000:3.0:1789760797.086391:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.086404:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.086415:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.086423:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.086429:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.086439:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.086458:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.086466:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036a44e08/0x69dc3c3d61d206c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.086741:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515847040:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34271us (38312us total) trans 4294973913 rc 0/0 00000100:00100000:3.0:1789760797.086753:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6625 00000100:00100000:3.0:1789760797.086764:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775471740077 time 775517708412 state 0x2 ref 0 00000100:00100000:0.0:1789760797.097087:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.097524:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847168 00000100:00100000:2.0:1789760797.097535:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847168 00000100:00100000:2.0:1789760797.097603:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775471740077 time 775517708412 state 0x0 ref 1 00000100:00100000:2.0:1789760797.097619:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775517968828 time 775517708412 state 0x1 ref 1 00000100:00100000:2.0:1789760797.097655:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775517968828 time 775562943423 state 0x0 ref 1 00000100:00100000:2.0:1789760797.097668:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515847168:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.097757:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27021312, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760797.097766:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.097781:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760797.097785:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.097791:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.097804:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.138010:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27021312 -> 27025408 00010000:00010000:1.0:1789760797.138029:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.138041:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.138050:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.138057:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760797.138063:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.138071:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.138089:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.138093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973914 is committed 00010000:00010000:1.0:1789760797.138096:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e36066a/0x69dc3c3d61d206cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.138185:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805637100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515847168:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40516us (41103us total) trans 4294973914 rc 0/0 00000100:00100000:1.0:1789760797.138196:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6626 00000100:00100000:1.0:1789760797.138207:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775517968828 time 775562943423 state 0x2 ref 0 00000100:00100000:1.0:1789760797.141628:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.141765:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847296 00000100:00100000:2.0:1789760797.141774:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847296 00000100:00100000:2.0:1789760797.141820:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775517968828 time 775562943423 state 0x0 ref 1 00000100:00100000:2.0:1789760797.141832:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775563203839 time 775562943423 state 0x1 ref 1 00000100:00100000:2.0:1789760797.141922:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775563203839 time 775607210469 state 0x0 ref 1 00000100:00100000:2.0:1789760797.141934:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515847296:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.141986:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27025408, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760797.141995:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.142013:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760797.142017:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.142025:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.142038:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.186113:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973915 is committed 00002000:00010000:3.0:1789760797.187749:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27025408 -> 27029504 00010000:00010000:3.0:1789760797.187769:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.187782:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.187790:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.187797:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.187803:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.187813:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.187834:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.187949:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d206d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.188050:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515847296:12345-192.168.203.46@tcp:4:dd.0 Request processed in 46113us (46428us total) trans 4294973915 rc 0/0 00000100:00100000:3.0:1789760797.188060:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6627 00000100:00100000:3.0:1789760797.188069:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775563203839 time 775607210469 state 0x2 ref 0 00000100:00100000:0.0:1789760797.191446:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.191541:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847424 00000100:00100000:2.0:1789760797.191552:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847424 00000100:00100000:2.0:1789760797.191614:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775563203839 time 775607210469 state 0x0 ref 1 00000100:00100000:2.0:1789760797.191629:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775607470885 time 775607210469 state 0x1 ref 1 00000100:00100000:2.0:1789760797.191657:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775607470885 time 775656946733 state 0x0 ref 1 00000100:00100000:2.0:1789760797.191669:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515847424:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.191745:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27029504, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760797.191755:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.191771:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760797.191776:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.191783:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.191797:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.236648:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973916 is committed 00002000:00010000:0.0:1789760797.236803:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27029504 -> 27033600 00010000:00010000:0.0:1789760797.236820:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.236833:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.236910:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.236918:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760797.236923:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.236932:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.236950:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.236957:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000051c284a2/0x69dc3c3d61d206db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.237086:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515847424:12345-192.168.203.46@tcp:4:dd.0 Request processed in 45414us (45645us total) trans 4294973916 rc 0/0 00000100:00100000:0.0:1789760797.237096:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6628 00000100:00100000:0.0:1789760797.237105:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775607470885 time 775656946733 state 0x2 ref 0 00000100:00100000:0.0:1789760797.240427:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.240580:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847552 00000100:00100000:2.0:1789760797.240590:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847552 00000100:00100000:2.0:1789760797.240646:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775607470885 time 775656946733 state 0x0 ref 1 00000100:00100000:2.0:1789760797.240662:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775657207149 time 775656946733 state 0x1 ref 1 00000100:00100000:2.0:1789760797.240752:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775657207149 time 775706039856 state 0x0 ref 1 00000100:00100000:2.0:1789760797.240766:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515847552:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.240815:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27033600, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760797.240824:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.240906:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760797.240913:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.240920:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.240934:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.269081:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27033600 -> 27037696 00010000:00010000:3.0:1789760797.269100:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.269114:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.269122:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.269130:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.269135:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.269145:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.269165:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.269173:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004dd26d00/0x69dc3c3d61d206e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.269267:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515847552:12345-192.168.203.46@tcp:4:dd.0 Request processed in 28498us (28845us total) trans 4294973917 rc 0/0 00000100:00100000:3.0:1789760797.269282:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6629 00000100:00100000:3.0:1789760797.269291:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775657207149 time 775706039856 state 0x2 ref 0 00000100:00100000:0.0:1789760797.273135:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760797.273225:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847680 00000100:00100000:3.0:1789760797.273237:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847680 00000100:00100000:3.0:1789760797.273295:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775657207149 time 775706039856 state 0x0 ref 1 00000100:00100000:3.0:1789760797.273311:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775706300272 time 775706039856 state 0x1 ref 1 00000100:00100000:3.0:1789760797.273341:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775706300272 time 775738629712 state 0x0 ref 1 00000100:00100000:3.0:1789760797.273354:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515847680:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.273401:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27037696, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.273409:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.273424:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760797.273428:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.273434:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.273447:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.274186:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973917 is committed 00002000:00010000:3.0:1789760797.303616:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27037696 -> 27041792 00010000:00010000:3.0:1789760797.303635:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.303648:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.303657:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.303664:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.303668:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.303675:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.303693:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.303701:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.303824:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515847680:12345-192.168.203.46@tcp:4:dd.0 Request processed in 30466us (30692us total) trans 4294973918 rc 0/0 00000100:00100000:3.0:1789760797.303835:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6630 00000100:00100000:3.0:1789760797.303845:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775706300272 time 775738629712 state 0x2 ref 0 00000100:00100000:2.0:1789760797.307142:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760797.307232:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847808 00000100:00100000:3.0:1789760797.307242:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847808 00000100:00100000:3.0:1789760797.307297:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775706300272 time 775738629712 state 0x0 ref 1 00000100:00100000:3.0:1789760797.307312:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775738890128 time 775738629712 state 0x1 ref 1 00000100:00100000:3.0:1789760797.307345:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775738890128 time 775772634707 state 0x0 ref 1 00000100:00100000:3.0:1789760797.307356:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515847808:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.307401:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27041792, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.307410:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.307426:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760797.307431:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.307437:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.307451:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.311093:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973918 is committed 00000001:00080000:1.0:1789760797.331172:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973919 is committed 00002000:00010000:3.0:1789760797.331379:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27041792 -> 27045888 00010000:00010000:3.0:1789760797.331397:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.331411:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.331426:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.331434:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.331439:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.331448:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.331469:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.331478:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d206f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.331589:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515847808:12345-192.168.203.46@tcp:4:dd.0 Request processed in 24229us (24451us total) trans 4294973919 rc 0/0 00000100:00100000:3.0:1789760797.331600:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6631 00000100:00100000:3.0:1789760797.331610:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775738890128 time 775772634707 state 0x2 ref 0 00000100:00100000:2.0:1789760797.335896:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760797.336602:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515847936 00000100:00100000:1.0:1789760797.336613:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515847936 00000100:00100000:1.0:1789760797.336749:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775738890128 time 775772634707 state 0x0 ref 1 00000100:00100000:1.0:1789760797.336766:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775772895123 time 775772634707 state 0x1 ref 1 00000100:00100000:3.0:1789760797.336796:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775772895123 time 775802083053 state 0x0 ref 1 00000100:00100000:3.0:1789760797.336811:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515847936:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.336865:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27045888, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.336874:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.336891:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760797.336896:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.336902:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.336916:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.369375:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973920 is committed 00002000:00010000:3.0:1789760797.369463:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27045888 -> 27049984 00010000:00010000:3.0:1789760797.369478:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369561:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369579:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369586:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.369591:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369600:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369619:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369627:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d206f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.369781:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515847936:12345-192.168.203.46@tcp:4:dd.0 Request processed in 32968us (33889us total) trans 4294973920 rc 0/0 00000100:00100000:3.0:1789760797.369790:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6632 00000100:00100000:3.0:1789760797.369800:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775772895123 time 775802083053 state 0x2 ref 0 00000100:00100000:0.0:1789760797.375402:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760797.375646:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848064 00000100:00100000:1.0:1789760797.375656:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848064 00000100:00100000:1.0:1789760797.375815:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775772895123 time 775802083053 state 0x0 ref 1 00000100:00100000:1.0:1789760797.375831:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775802343469 time 775802083053 state 0x1 ref 1 00000100:00100000:1.0:1789760797.375888:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775802343469 time 775841176649 state 0x0 ref 1 00000100:00100000:1.0:1789760797.375902:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515848064:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760797.375952:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27049984, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:1.0:1789760797.375960:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.375975:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760797.375979:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.375986:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.375999:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.441991:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973921 is committed 00002000:00010000:3.0:1789760797.442198:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27049984 -> 27054080 00010000:00010000:3.0:1789760797.442216:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.442229:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.442238:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.442246:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767c00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.442250:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.442265:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.442284:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.442292:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e94766a/0x69dc3c3d61d206fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.442420:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515848064:12345-192.168.203.46@tcp:4:dd.0 Request processed in 66514us (67022us total) trans 4294973921 rc 0/0 00000100:00100000:3.0:1789760797.442430:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6633 00000100:00100000:3.0:1789760797.442439:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775802343469 time 775841176649 state 0x2 ref 0 00000100:00100000:2.0:1789760797.446303:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760797.446401:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848192 00000100:00100000:3.0:1789760797.446411:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848192 00000100:00100000:3.0:1789760797.446528:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775802343469 time 775841176649 state 0x0 ref 1 00000100:00100000:3.0:1789760797.446547:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775841437065 time 775841176649 state 0x1 ref 1 00000100:00100000:3.0:1789760797.446610:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775841437065 time 775911898749 state 0x0 ref 1 00000100:00100000:3.0:1789760797.446624:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515848192:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.446679:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27054080, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.446689:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.446725:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760797.446730:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.446736:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.446750:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760797.481236:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27054080 -> 27058176 00010000:00010000:0.0:1789760797.481257:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.481271:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.481280:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.481288:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36ce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760797.481293:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.481302:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.481322:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.481330:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038fba7dc/0x69dc3c3d61d20705 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.481422:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2365:x1876699515848192:12345-192.168.203.46@tcp:4:dd.0 Request processed in 34795us (35121us total) trans 4294973922 rc 0/0 00000100:00100000:0.0:1789760797.481432:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6634 00000100:00100000:0.0:1789760797.481442:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775841437065 time 775911898749 state 0x2 ref 0 00000001:00080000:1.0:1789760797.481575:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973922 is committed 00000100:00100000:3.0:1789760797.491627:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760797.491702:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848320 00000100:00100000:1.0:1789760797.491800:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848320 00000100:00100000:1.0:1789760797.491968:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775841437065 time 775911898749 state 0x0 ref 1 00000100:00100000:1.0:1789760797.491988:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775912159165 time 775911898749 state 0x1 ref 1 00000100:00100000:1.0:1789760797.492016:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775912159165 time 775957305909 state 0x0 ref 1 00000100:00100000:1.0:1789760797.492029:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515848320:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760797.492078:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27058176, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:1.0:1789760797.492087:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.492103:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760797.492108:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.492114:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.492127:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.532337:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973923 is committed 00002000:00010000:3.0:1789760797.532440:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27058176 -> 27062272 00010000:00010000:3.0:1789760797.532456:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.532469:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.532477:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.532494:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.532498:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.532508:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.532532:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.532538:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba1ddc3/0x69dc3c3d61d2070c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.532667:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80620d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515848320:12345-192.168.203.46@tcp:4:dd.0 Request processed in 40636us (41044us total) trans 4294973923 rc 0/0 00000100:00100000:3.0:1789760797.532678:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6635 00000100:00100000:3.0:1789760797.532687:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775912159165 time 775957305909 state 0x2 ref 0 00000100:00100000:3.0:1789760797.539357:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760797.543480:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848448 00000100:00100000:1.0:1789760797.544199:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848448 00000100:00100000:1.0:1789760797.544265:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775912159165 time 775957305909 state 0x0 ref 1 00000100:00100000:1.0:1789760797.544282:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775957566325 time 775957305909 state 0x1 ref 1 00000100:00100000:1.0:1789760797.544312:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775957566325 time 776009601142 state 0x0 ref 1 00000100:00100000:1.0:1789760797.544326:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515848448:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760797.544390:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27062272, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:1.0:1789760797.544399:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.544416:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.544421:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.544427:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.544444:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.596298:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973924 is committed 00002000:00010000:3.0:1789760797.596526:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27062272 -> 27066368 00010000:00010000:3.0:1789760797.596541:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.596553:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.596572:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.596579:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.596584:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.596607:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.596628:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.596635:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20713 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.596766:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c064e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515848448:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52437us (57412us total) trans 4294973924 rc 0/0 00000100:00100000:3.0:1789760797.596777:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6636 00000100:00100000:3.0:1789760797.596787:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775957566325 time 776009601142 state 0x2 ref 0 00000100:00100000:0.0:1789760797.604309:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.607029:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848576 00000100:00100000:2.0:1789760797.607042:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848576 00000100:00100000:2.0:1789760797.607105:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 775957566325 time 776009601142 state 0x0 ref 1 00000100:00100000:2.0:1789760797.607120:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776009861558 time 776009601142 state 0x1 ref 1 00000100:00100000:2.0:1789760797.607144:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776009861558 time 776072433738 state 0x0 ref 1 00000100:00100000:2.0:1789760797.607158:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937ca4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515848576:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.607207:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27066368, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:2.0:1789760797.607216:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.607231:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760797.607235:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.607241:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.607255:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.661295:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973925 is committed 00002000:00010000:3.0:1789760797.661436:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27066368 -> 27070464 00010000:00010000:3.0:1789760797.661452:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.661464:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.661473:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.661480:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.661548:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.661559:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.661587:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.661595:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2071a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.661748:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937ca4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515848576:12345-192.168.203.46@tcp:4:dd.0 Request processed in 54552us (57408us total) trans 4294973925 rc 0/0 00000100:00100000:3.0:1789760797.661758:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6637 00000100:00100000:3.0:1789760797.661768:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776009861558 time 776072433738 state 0x2 ref 0 00000100:00100000:0.0:1789760797.670879:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760797.670969:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848704 00000100:00100000:3.0:1789760797.670979:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848704 00000100:00100000:3.0:1789760797.671034:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776009861558 time 776072433738 state 0x0 ref 1 00000100:00100000:3.0:1789760797.671050:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776072694154 time 776072433738 state 0x1 ref 1 00000100:00100000:3.0:1789760797.671084:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776072694154 time 776136372326 state 0x0 ref 1 00000100:00100000:3.0:1789760797.671095:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515848704:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.671143:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27070464, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.671153:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.671169:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.671173:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.671179:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.671192:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.709132:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973926 is committed 00002000:00010000:0.0:1789760797.709210:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27070464 -> 27074560 00010000:00010000:0.0:1789760797.709226:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.709237:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.709245:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.709251:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36dc00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760797.709256:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.709264:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.709283:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.709290:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001478799e/0x69dc3c3d61d20721 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.709700:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515848704:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38601us (38824us total) trans 4294973926 rc 0/0 00000100:00100000:0.0:1789760797.709761:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6638 00000100:00100000:0.0:1789760797.709771:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776072694154 time 776136372326 state 0x2 ref 0 00000100:00100000:0.0:1789760797.718660:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760797.718907:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848832 00000100:00100000:3.0:1789760797.718918:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848832 00000100:00100000:3.0:1789760797.718974:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776072694154 time 776136372326 state 0x0 ref 1 00000100:00100000:3.0:1789760797.718988:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776136632742 time 776136372326 state 0x1 ref 1 00000100:00100000:3.0:1789760797.719022:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776136632742 time 776184311045 state 0x0 ref 1 00000100:00100000:3.0:1789760797.719034:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515848832:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.719086:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27074560, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.719095:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.719112:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.719116:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.719122:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.719136:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.805957:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27074560 -> 27078656 00010000:00010000:3.0:1789760797.805978:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.805992:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.806001:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.806008:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.806014:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.806023:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.806042:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.806050:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20728 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.806146:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2362:x1876699515848832:12345-192.168.203.46@tcp:4:dd.0 Request processed in 87109us (87491us total) trans 4294973927 rc 0/0 00000100:00100000:3.0:1789760797.806156:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6639 00000100:00100000:3.0:1789760797.806167:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776136632742 time 776184311045 state 0x2 ref 0 00000001:00080000:2.0:1789760797.806916:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973927 is committed 00000100:00100000:0.0:1789760797.815827:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760797.815910:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515848960 00000100:00100000:3.0:1789760797.815919:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515848960 00000100:00100000:3.0:1789760797.815970:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776136632742 time 776184311045 state 0x0 ref 1 00000100:00100000:3.0:1789760797.815984:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776184571461 time 776184311045 state 0x1 ref 1 00000100:00100000:3.0:1789760797.816012:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776184571461 time 776281301906 state 0x0 ref 1 00000100:00100000:3.0:1789760797.816024:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515848960:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760797.816070:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27078656, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xd007 00010000:00010000:3.0:1789760797.816079:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.816094:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.816098:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.816104:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.816118:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.860931:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973928 is committed 00002000:00010000:3.0:1789760797.862899:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27078656 -> 27082752 00002000:00010000:3.0:1789760797.862906:0:6713:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789760797 -> 1789760798 00002000:00010000:3.0:1789760797.862909:0:6713:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789760797 -> 1789760798 00010000:00010000:3.0:1789760797.862922:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.862934:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.862943:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.862950:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.862954:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.862963:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.862982:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.862990:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d2072f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.863145:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515848960:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47118us (47322us total) trans 4294973928 rc 0/0 00000100:00100000:3.0:1789760797.863157:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6640 00000100:00100000:3.0:1789760797.863166:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776184571461 time 776281301906 state 0x2 ref 0 00000100:00100000:0.0:1789760797.874827:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760797.875328:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849088 00000100:00100000:1.0:1789760797.875339:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849088 00000100:00100000:1.0:1789760797.875399:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776184571461 time 776281301906 state 0x0 ref 1 00000100:00100000:1.0:1789760797.875416:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776281562322 time 776281301906 state 0x1 ref 1 00000100:00100000:1.0:1789760797.875449:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776281562322 time 776340738012 state 0x0 ref 1 00000100:00100000:1.0:1789760797.875463:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515849088:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760797.875520:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27082752, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:1.0:1789760797.875529:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.875551:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.875556:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.875571:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.875585:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.920344:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973929 is committed 00002000:00010000:3.0:1789760797.923118:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27082752 -> 27086848 00010000:00010000:3.0:1789760797.923138:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.923152:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.923160:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.923167:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.923172:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.923181:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.923202:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.923216:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000521c7cf9/0x69dc3c3d61d20736 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.923321:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c9bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515849088:12345-192.168.203.46@tcp:4:dd.0 Request processed in 47856us (48501us total) trans 4294973929 rc 0/0 00000100:00100000:3.0:1789760797.923330:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6641 00000100:00100000:3.0:1789760797.923340:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776281562322 time 776340738012 state 0x2 ref 0 00000100:00100000:2.0:1789760797.928414:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760797.929216:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849216 00000100:00100000:1.0:1789760797.929226:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849216 00000100:00100000:1.0:1789760797.929285:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776281562322 time 776340738012 state 0x0 ref 1 00000100:00100000:1.0:1789760797.929300:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776340998428 time 776340738012 state 0x1 ref 1 00000100:00100000:1.0:1789760797.929333:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776340998428 time 776394621556 state 0x0 ref 1 00000100:00100000:1.0:1789760797.929345:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515849216:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760797.929391:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27086848, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:1.0:1789760797.929400:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.929414:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.929419:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.929424:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.929441:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.981496:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973930 is committed 00002000:00010000:3.0:1789760797.981719:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27086848 -> 27090944 00010000:00010000:3.0:1789760797.981732:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.981742:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.981747:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.981752:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c767000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760797.981755:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.981761:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.981783:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.981788:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071b45b77/0x69dc3c3d61d2073d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.981866:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805409180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515849216:12345-192.168.203.46@tcp:4:dd.0 Request processed in 52518us (53456us total) trans 4294973930 rc 0/0 00000100:00100000:3.0:1789760797.981877:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6642 00000100:00100000:3.0:1789760797.981886:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776340998428 time 776394621556 state 0x2 ref 0 00000100:00100000:0.0:1789760797.985127:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760797.985610:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849344 00000100:00100000:2.0:1789760797.985622:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849344 00000100:00100000:2.0:1789760797.985672:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776340998428 time 776394621556 state 0x0 ref 1 00000100:00100000:2.0:1789760797.985687:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776394881972 time 776394621556 state 0x1 ref 1 00000100:00100000:2.0:1789760797.985751:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776394881972 time 776451039297 state 0x0 ref 1 00000100:00100000:2.0:1789760797.985763:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515849344:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760797.985808:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27090944, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760797.985817:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.985841:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760797.985845:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.985852:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.985865:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.013970:0:5842:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760798.022811:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973931 is committed 00002000:00010000:3.0:1789760798.023162:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27090944 -> 27095040 00010000:00010000:3.0:1789760798.023179:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.023192:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.023202:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.023210:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760798.023215:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.023229:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.023247:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.023255:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d20744 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.023353:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2362:x1876699515849344:12345-192.168.203.46@tcp:4:dd.0 Request processed in 37586us (38230us total) trans 4294973931 rc 0/0 00000100:00100000:3.0:1789760798.023364:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6643 00000100:00100000:3.0:1789760798.023374:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776394881972 time 776451039297 state 0x2 ref 0 00000100:00100000:0.0:1789760798.032318:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760798.032623:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849472 00000100:00100000:2.0:1789760798.032631:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849472 00000100:00080000:0.0:1789760798.032678:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760798.032688:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776394881972 time 776451039297 state 0x0 ref 1 00000100:00100000:2.0:1789760798.032703:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776451299713 time 776451039297 state 0x1 ref 1 00000100:00100000:2.0:1789760798.032764:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776451299713 time 776498052900 state 0x0 ref 1 00000100:00100000:2.0:1789760798.032777:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515849472:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760798.032833:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27095040, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760798.032842:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.032856:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760798.032860:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.032867:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.032880:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.085925:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27095040 -> 27099136 00000001:00080000:0.0:1789760798.085948:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973932 is committed 00010000:00010000:3.0:1789760798.086021:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.086034:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.086043:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.086051:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103a00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760798.086056:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.086064:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.086083:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.086091:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012c3f2e4/0x69dc3c3d61d2074b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.086183:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae92fa84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515849472:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53403us (53871us total) trans 4294973932 rc 0/0 00000100:00100000:3.0:1789760798.086193:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6644 00000100:00100000:3.0:1789760798.086274:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776451299713 time 776498052900 state 0x2 ref 0 00000100:00100000:0.0:1789760798.092071:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760798.092165:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849600 00000100:00100000:3.0:1789760798.092176:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849600 00000100:00100000:3.0:1789760798.092227:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776451299713 time 776498052900 state 0x0 ref 1 00000100:00100000:3.0:1789760798.092241:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776498313316 time 776498052900 state 0x1 ref 1 00000100:00100000:3.0:1789760798.092308:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776498313316 time 776557596479 state 0x0 ref 1 00000100:00100000:3.0:1789760798.092322:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515849600:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760798.092367:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27099136, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:3.0:1789760798.092377:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.092393:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760798.092397:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.092403:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.092418:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.127182:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973933 is committed 00002000:00010000:2.0:1789760798.127318:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27099136 -> 27103232 00010000:00010000:2.0:1789760798.127333:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.127351:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.127361:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.127369:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36c800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760798.127374:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.127383:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.127404:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.127413:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebdea759/0x69dc3c3d61d20752 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.127606:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515849600:12345-192.168.203.46@tcp:4:dd.0 Request processed in 35282us (35539us total) trans 4294973933 rc 0/0 00000100:00100000:2.0:1789760798.127618:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6645 00000100:00100000:2.0:1789760798.127628:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776498313316 time 776557596479 state 0x2 ref 0 00000100:00100000:0.0:1789760798.132931:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760798.133072:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849728 00000100:00100000:3.0:1789760798.133083:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849728 00000100:00100000:3.0:1789760798.133135:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776498313316 time 776557596479 state 0x0 ref 1 00000100:00100000:3.0:1789760798.133151:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776557856895 time 776557596479 state 0x1 ref 1 00000100:00100000:3.0:1789760798.133183:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776557856895 time 776598471867 state 0x0 ref 1 00000100:00100000:3.0:1789760798.133195:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515849728:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:3.0:1789760798.133255:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27103232, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:3.0:1789760798.133265:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.133281:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760798.133285:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.133292:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.133306:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.176418:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27103232 -> 27107328 00010000:00010000:1.0:1789760798.176439:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176452:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176461:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176469:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae92f36d200 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760798.176474:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176483:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176503:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.176511:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098de3f0a/0x69dc3c3d61d20759 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.177035:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae937c5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515849728:12345-192.168.203.46@tcp:4:dd.0 Request processed in 43836us (44109us total) trans 4294973934 rc 0/0 00000100:00100000:1.0:1789760798.177047:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6646 00000100:00100000:1.0:1789760798.177057:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776557856895 time 776598471867 state 0x2 ref 0 00000001:00080000:1.0:1789760798.177108:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973934 is committed 00000100:00100000:0.0:1789760798.187495:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:3.0:1789760798.189777:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849856 00000100:00100000:3.0:1789760798.189792:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849856 00000100:00100000:3.0:1789760798.189860:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776557856895 time 776598471867 state 0x0 ref 1 00000100:00100000:3.0:1789760798.189876:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776598732283 time 776598471867 state 0x1 ref 1 00000100:00100000:3.0:1789760798.189914:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776598732283 time 776655202629 state 0x0 ref 1 00000100:00100000:3.0:1789760798.189927:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515849856:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760798.190008:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27107328, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760798.190020:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190038:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760798.190043:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.190049:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190063:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.207871:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805408000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544423552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.207937:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760798.207992:0:3322:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760798.207999:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805409880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544423168:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.208027:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760798.208046:0:3322:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805408380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544423680:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.208072:0:3322:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760798.208098:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805409f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544423424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.208120:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760798.208137:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760798.208142:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805409c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544423296:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.208162:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760798.208178:0:3321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff8ae805408e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544423808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.208197:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760798.208223:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423552 00000100:00100000:2.0:1789760798.208230:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423552 00000100:00100000:2.0:1789760798.208277:0:6708:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423808 00000100:00100000:2.0:1789760798.208280:0:6708:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423808 00000100:00100000:2.0:1789760798.208304:0:6708:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1789760798.208312:0:6708:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544423552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760798.208544:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423168 00000100:00100000:1.0:1789760798.208549:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423168 00000100:00100000:1.0:1789760798.208587:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760798.208594:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544423168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760798.209673:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423680 00000100:00100000:3.0:1789760798.209680:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423680 00000100:00100000:3.0:1789760798.209732:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423424 00000100:00100000:3.0:1789760798.209735:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423424 00000100:00100000:3.0:1789760798.209750:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699544423296 00000100:00100000:3.0:1789760798.209753:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699544423296 00000100:00100000:3.0:1789760798.209766:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789760798.209772:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544423680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760798.210539:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:850d5753-05fb-4719-a6af-8cf215b926a5+19:3322:x1876699544423168:12345-0@lo:400:kworker.0 Request processed in 1943us (2508us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760798.210549:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760798.210619:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805409880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:850d5753-05fb-4719-a6af-8cf215b926a5:3322:1876699544423168:0@lo:400:kworker.0 00000100:00100000:0.0:1789760798.211000:0:8226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1789760798.211011:0:8226:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544423808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760798.211190:0:8226:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3321:x1876699544423808:12345-0@lo:400:kworker.0 Request processed in 182us (2991us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760798.211196:0:8226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00080000:0.0:1789760798.211221:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760798.211226:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760798.211228:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760798.211252:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805408e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3321:1876699544423808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760798.212004:0:6708:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae80540ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3322:x1876699544423552:12345-0@lo:400:kworker.0 Request processed in 3690us (4064us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760798.212013:0:6708:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:1.0:1789760798.212641:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805408000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3322:1876699544423552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760798.215429:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3322:x1876699544423680:12345-0@lo:400:kworker.0 Request processed in 5647us (7346us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760798.215442:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789760798.215527:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1789760798.215540:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544423424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760798.215605:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1876699544423424:12345-0@lo:400:kworker.0 Request processed in 70us (7483us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760798.215611:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1789760798.215623:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1789760798.215629:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544423296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760798.215653:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1876699544423296:12345-0@lo:400:kworker.0 Request processed in 24us (7489us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760798.215658:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1789760798.215723:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805409c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3321:1876699544423296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760798.215735:0:3321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805409f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3321:1876699544423424:0@lo:400:kworker.0 00000100:00100000:1.0:1789760798.217015:0:3322:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff8ae805408380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3322:1876699544423680:0@lo:400:kworker.0 00000001:00080000:3.0:1789760798.228662:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973935 is committed 00002000:00010000:2.0:1789760798.228750:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27107328 -> 27111424 00010000:00010000:2.0:1789760798.228765:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.228774:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.228780:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.228785:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760798.228788:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.228795:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.228810:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.228818:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d20760 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.228894:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b669880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515849856:12345-192.168.203.46@tcp:4:dd.0 Request processed in 38962us (41401us total) trans 4294973935 rc 0/0 00000100:00100000:2.0:1789760798.228904:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6647 00000100:00100000:2.0:1789760798.228913:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776598732283 time 776655202629 state 0x2 ref 0 00000100:00100000:2.0:1789760798.231660:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760798.231754:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515849984 00000100:00100000:0.0:1789760798.231762:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515849984 00000100:00100000:0.0:1789760798.231812:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776598732283 time 776655202629 state 0x0 ref 1 00000100:00100000:0.0:1789760798.231828:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776655463045 time 776655202629 state 0x1 ref 1 00000100:00100000:0.0:1789760798.231854:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776655463045 time 776697143086 state 0x0 ref 1 00000100:00100000:0.0:1789760798.231867:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515849984:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760798.231912:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27111424, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:0.0:1789760798.231921:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.231936:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760798.231941:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.231947:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.231961:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.264645:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973936 is committed 00002000:00010000:2.0:1789760798.264762:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27111424 -> 27115520 00010000:00010000:2.0:1789760798.264778:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.264792:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.264803:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.264810:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bce00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760798.264815:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.264823:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.264848:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.264856:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d20767 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.264956:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515849984:12345-192.168.203.46@tcp:4:dd.0 Request processed in 33087us (33300us total) trans 4294973936 rc 0/0 00000100:00100000:2.0:1789760798.264967:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6648 00000100:00100000:2.0:1789760798.264977:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776655463045 time 776697143086 state 0x2 ref 0 00000100:00100000:0.0:1789760798.269413:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760798.269479:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850112 00000100:00100000:2.0:1789760798.269489:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850112 00000100:00100000:2.0:1789760798.269541:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776655463045 time 776697143086 state 0x0 ref 1 00000100:00100000:2.0:1789760798.269568:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776697403502 time 776697143086 state 0x1 ref 1 00000100:00100000:2.0:1789760798.269599:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776697403502 time 776734887727 state 0x0 ref 1 00000100:00100000:2.0:1789760798.269612:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850112:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760798.269660:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27115520, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760798.269668:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.269684:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760798.269688:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.269695:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.269753:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.300358:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973937 is committed 00002000:00010000:0.0:1789760798.300601:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27115520 -> 27119616 00010000:00010000:0.0:1789760798.300617:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300629:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300639:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300645:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760798.300650:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300658:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300678:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300686:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aab12aca/0x69dc3c3d61d2076e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.300889:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c023480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850112:12345-192.168.203.46@tcp:4:dd.0 Request processed in 31274us (31479us total) trans 4294973937 rc 0/0 00000100:00100000:0.0:1789760798.300900:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6649 00000100:00100000:0.0:1789760798.300909:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776697403502 time 776734887727 state 0x2 ref 0 00000100:00100000:1.0:1789760798.304149:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760798.304285:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850240 00000100:00100000:2.0:1789760798.304295:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850240 00000100:00100000:2.0:1789760798.304344:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776697403502 time 776734887727 state 0x0 ref 1 00000100:00100000:2.0:1789760798.304358:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776735148143 time 776734887727 state 0x1 ref 1 00000100:00100000:2.0:1789760798.304413:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776735148143 time 776769702886 state 0x0 ref 1 00000100:00100000:2.0:1789760798.304425:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850240:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760798.304474:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27119616, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760798.304493:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.304507:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760798.304511:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.304518:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.304531:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760798.349150:0:13070:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27119616 -> 27123712 00010000:00010000:0.0:1789760798.349171:0:13070:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.349190:0:13070:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.349199:0:13070:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.349207:0:13070:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c102600 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760798.349212:0:13070:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.349221:0:13070:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.349241:0:13070:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.349249:0:13070:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004244476b/0x69dc3c3d61d20775 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.349341:0:13070:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515850240:12345-192.168.203.46@tcp:4:dd.0 Request processed in 44912us (45196us total) trans 4294973938 rc 0/0 00000100:00100000:0.0:1789760798.349352:0:13070:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6650 00000100:00100000:0.0:1789760798.349363:0:13070:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776735148143 time 776769702886 state 0x2 ref 0 00000001:00080000:1.0:1789760798.350677:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973938 is committed 00000100:00100000:2.0:1789760798.353925:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760798.354071:0:6712:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850368 00000100:00100000:0.0:1789760798.354081:0:6712:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850368 00000100:00100000:0.0:1789760798.354135:0:6712:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776735148143 time 776769702886 state 0x0 ref 1 00000100:00100000:0.0:1789760798.354151:0:6712:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776769963302 time 776769702886 state 0x1 ref 1 00000100:00100000:0.0:1789760798.354179:0:6712:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776769963302 time 776819468437 state 0x0 ref 1 00000100:00100000:0.0:1789760798.354192:0:6712:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515850368:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760798.354239:0:6712:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27123712, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:0.0:1789760798.354248:0:6712:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354264:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760798.354268:0:6712:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.354275:0:6712:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354293:0:6712:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.405280:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973939 is committed 00002000:00010000:0.0:1789760798.405453:0:6712:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27123712 -> 27127808 00010000:00010000:0.0:1789760798.405468:0:6712:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.405490:0:6712:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.405505:0:6712:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.405511:0:6712:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bd800 already unused, calling callback (ffffffffc1708350) 00010000:00010000:0.0:1789760798.405516:0:6712:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.405524:0:6712:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.405542:0:6712:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.405550:0:6712:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3444ac3/0x69dc3c3d61d2077c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6712 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.405661:0:6712:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515850368:12345-192.168.203.46@tcp:4:dd.0 Request processed in 51465us (51740us total) trans 4294973939 rc 0/0 00000100:00100000:0.0:1789760798.405671:0:6712:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6651 00000100:00100000:0.0:1789760798.405680:0:6712:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776769963302 time 776819468437 state 0x2 ref 0 00000100:00100000:2.0:1789760798.411283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760798.411417:0:6713:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850496 00000100:00100000:1.0:1789760798.411426:0:6713:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850496 00000100:00100000:1.0:1789760798.411494:0:6713:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776769963302 time 776819468437 state 0x0 ref 1 00000100:00100000:1.0:1789760798.411512:0:6713:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776819728853 time 776819468437 state 0x1 ref 1 00000100:00100000:1.0:1789760798.411540:0:6713:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776819728853 time 776876829533 state 0x0 ref 1 00000100:00100000:1.0:1789760798.411553:0:6713:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850496:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:1.0:1789760798.411608:0:6713:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27127808, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:1.0:1789760798.411617:0:6713:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.411633:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760798.411637:0:6713:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.411643:0:6713:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.411658:0:6713:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.475819:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973940 is committed 00002000:00010000:2.0:1789760798.475884:0:6713:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27127808 -> 27131904 00010000:00010000:2.0:1789760798.475909:0:6713:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.475922:0:6713:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.475930:0:6713:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.475937:0:6713:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c766e00 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760798.475942:0:6713:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.475951:0:6713:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.475979:0:6713:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.475987:0:6713:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4179a80/0x69dc3c3d61d20783 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6713 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.476088:0:6713:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804c4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850496:12345-192.168.203.46@tcp:4:dd.0 Request processed in 64532us (64809us total) trans 4294973940 rc 0/0 00000100:00100000:2.0:1789760798.476098:0:6713:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6652 00000100:00100000:2.0:1789760798.476107:0:6713:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776819728853 time 776876829533 state 0x2 ref 0 00000100:00100000:1.0:1789760798.483971:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760798.484035:0:6711:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850624 00000100:00100000:2.0:1789760798.484044:0:6711:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850624 00000100:00100000:2.0:1789760798.484097:0:6711:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776819728853 time 776876829533 state 0x0 ref 1 00000100:00100000:2.0:1789760798.484112:0:6711:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776877089949 time 776876829533 state 0x1 ref 1 00000100:00100000:2.0:1789760798.484146:0:6711:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776877089949 time 776949435036 state 0x0 ref 1 00000100:00100000:2.0:1789760798.484158:0:6711:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850624:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760798.484315:0:6711:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27131904, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760798.484325:0:6711:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.484341:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760798.484345:0:6711:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.484351:0:6711:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.484365:0:6711:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.546619:0:6711:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27131904 -> 27136000 00010000:00010000:1.0:1789760798.546639:0:6711:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.546652:0:6711:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.546662:0:6711:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.546670:0:6711:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:1.0:1789760798.546676:0:6711:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.546684:0:6711:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.546704:0:6711:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.546741:0:6711:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2409bca/0x69dc3c3d61d2078a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6711 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.546820:0:6711:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c02a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:048c2075-43b1-49c3-a1d8-2b3c1afa706e+7:2364:x1876699515850624:12345-192.168.203.46@tcp:4:dd.0 Request processed in 62660us (62855us total) trans 4294973941 rc 0/0 00000100:00100000:1.0:1789760798.546829:0:6711:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6653 00000100:00100000:1.0:1789760798.546839:0:6711:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776877089949 time 776949435036 state 0x2 ref 0 00000001:00080000:3.0:1789760798.546914:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973941 is committed 00000100:00100000:0.0:1789760798.553454:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:2.0:1789760798.553601:0:13071:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850752 00000100:00100000:2.0:1789760798.553611:0:13071:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850752 00000100:00100000:2.0:1789760798.553665:0:13071:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776877089949 time 776949435036 state 0x0 ref 1 00000100:00100000:2.0:1789760798.553681:0:13071:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776949695452 time 776949435036 state 0x1 ref 1 00000100:00100000:2.0:1789760798.553758:0:13071:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776949695452 time 777019046231 state 0x0 ref 1 00000100:00100000:2.0:1789760798.553772:0:13071:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850752:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:2.0:1789760798.553825:0:13071:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27136000, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:2.0:1789760798.553835:0:13071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.553852:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760798.553856:0:13071:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.553862:0:13071:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.553875:0:13071:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.606858:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973942 is committed 00002000:00010000:2.0:1789760798.606982:0:13071:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27136000 -> 27140096 00010000:00010000:2.0:1789760798.606996:0:13071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.607016:0:13071:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.607026:0:13071:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.607034:0:13071:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae93c103000 already unused, calling callback (ffffffffc1708350) 00010000:00010000:2.0:1789760798.607039:0:13071:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.607049:0:13071:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.607069:0:13071:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.607078:0:13071:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8c57a4c/0x69dc3c3d61d20791 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.607169:0:13071:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae804df0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2364:x1876699515850752:12345-192.168.203.46@tcp:4:dd.0 Request processed in 53394us (53722us total) trans 4294973942 rc 0/0 00000100:00100000:2.0:1789760798.607179:0:13071:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6654 00000100:00100000:2.0:1789760798.607188:0:13071:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776949695452 time 777019046231 state 0x2 ref 0 00000100:00100000:2.0:1789760798.614425:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760798.614582:0:13072:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515850880 00000100:00100000:0.0:1789760798.614592:0:13072:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515850880 00000100:00100000:0.0:1789760798.614652:0:13072:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 776949695452 time 777019046231 state 0x0 ref 1 00000100:00100000:0.0:1789760798.614667:0:13072:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 777019306647 time 777019046231 state 0x1 ref 1 00000100:00100000:0.0:1789760798.614701:0:13072:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 777019306647 time 777079990089 state 0x0 ref 1 00000100:00100000:0.0:1789760798.614739:0:13072:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515850880:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760798.614792:0:13072:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27140096, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:0.0:1789760798.614801:0:13072:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.614825:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760798.614831:0:13072:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.614837:0:13072:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.614856:0:13072:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.648545:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760798.648595:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760798.648673:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:1.0:1789760798.648703:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515851008 00000100:00100000:1.0:1789760798.648735:0:5849:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515851008 00000100:00100000:1.0:1789760798.648785:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 414 00000100:00100000:1.0:1789760798.648794:0:5849:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae90930bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515851008:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760798.648866:0:5849:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae90930bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:65ff60fc-9b44-48f8-988f-b0587af5cf52+13:2364:x1876699515851008:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 73us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760798.648873:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 414 00000100:00100000:1.0:1789760798.648903:0:10715:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515851136 00000100:00100000:1.0:1789760798.648907:0:10715:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515851136 00000100:00100000:3.0:1789760798.649020:0:8224:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515851264 00000100:00100000:3.0:1789760798.649024:0:8224:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515851264 00000100:00100000:1.0:1789760798.649082:0:10715:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 511 00000100:00100000:1.0:1789760798.649089:0:10715:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515851136:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:1.0:1789760798.649120:0:10715:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93c029c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:048c2075-43b1-49c3-a1d8-2b3c1afa706e+8:2365:x1876699515851136:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 31us (586us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760798.649126:0:10715:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 511 00000100:00100000:3.0:1789760798.658814:0:8224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.46@tcp, seq: 512 00000100:00100000:3.0:1789760798.658828:0:8224:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515851264:12345-192.168.203.46@tcp:400:kworker.0 00000100:00100000:3.0:1789760798.658904:0:8224:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae805408380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:cfe1b2c2-be6d-4234-9c05-42b712409d84+5:2364:x1876699515851264:12345-192.168.203.46@tcp:400:kworker.0 Request processed in 80us (10230us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760798.658910:0:8224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.46@tcp, seq: 512 00000100:00080000:1.0:1789760798.659350:0:5850:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760798.675787:0:6731:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973943 is committed 00002000:00010000:3.0:1789760798.676038:0:13072:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 27140096 -> 27144192 00010000:00010000:3.0:1789760798.676055:0:13072:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.676067:0:13072:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.676082:0:13072:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.676090:0:13072:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8ae9129bc400 already unused, calling callback (ffffffffc1708350) 00010000:00010000:3.0:1789760798.676094:0:13072:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.676103:0:13072:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.676121:0:13072:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.676128:0:13072:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee5ce068/0x69dc3c3d61d20798 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13072 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.676216:0:13072:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8ae93b323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2365:x1876699515850880:12345-192.168.203.46@tcp:4:dd.0 Request processed in 61476us (61801us total) trans 4294973943 rc 0/0 00000100:00100000:3.0:1789760798.676226:0:13072:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.46@tcp, seq: 6655 00000100:00100000:3.0:1789760798.676235:0:13072:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 777019306647 time 777079990089 state 0x2 ref 0 00000100:00100000:1.0:1789760798.682933:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.46@tcp (source: 12345-192.168.203.46@tcp) 00000100:00100000:0.0:1789760798.684305:0:13070:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699515851392 00000100:00100000:0.0:1789760798.684320:0:13070:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699515851392 00000100:00100000:0.0:1789760798.684384:0:13070:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 777019306647 time 777079990089 state 0x0 ref 1 00000100:00100000:0.0:1789760798.684400:0:13070:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 777080250505 time 777079990089 state 0x1 ref 1 00000100:00100000:0.0:1789760798.684433:0:13070:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff8ae90afe6000 key nid={192.168.203.46@tcp}&opcode={ost_write} rule@0xffff8ae9188e3200 gen 0 rate 15 depth 3 token 3 deadline 777080250505 time 777149721570 state 0x0 ref 1 00000100:00100000:0.0:1789760798.684450:0:13070:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8ae805409f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:048c2075-43b1-49c3-a1d8-2b3c1afa706e+5:2363:x1876699515851392:12345-192.168.203.46@tcp:4:dd.0 00002000:00010000:0.0:1789760798.684509:0:13070:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 27144192, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xd007 00010000:00010000:0.0:1789760798.684518:0:13070:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2079f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.684535:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760798.684539:0:13070:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.684546:0:13070:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2079f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.684569:0:13070:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efd35485/0x69dc3c3d61d2079f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13070 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 56504 lines, 56504 kept, 0 dropped, 0 bad.